View source for Trolley problem

{{Short description|Thought experiment in ethics}}
{{Redirect|The Trolley Problem|the television episode of the same name|The Trolley Problem (The Good Place)}}
[[File:Trolley Problem.svg|thumb|upright=1.35|One of the dilemmas included in the trolley problem: is it preferable to pull the lever to divert the runaway trolley onto the side track?]]

The '''trolley problem''' is a series of [[thought experiment]]s in [[ethics]], [[psychology]], and [[artificial intelligence]] involving stylized [[ethical dilemma]]s of whether to sacrifice one person to save a larger number.  The series usually begins with a [[Scenario (vehicular automation)|scenario]] in which a runaway [[tram]] or trolley is on course to collide with and kill a number of people (traditionally five) down the track, but a driver or bystander can intervene and divert the vehicle to kill just one person on a different track.  Then other variations of the runaway vehicle, and analogous life-and-death dilemmas (medical, judicial, etc.) are posed, each containing the option to either do nothing, in which case several people will be killed, or intervene and sacrifice one initially "safe" person to save the others.

Opinions on the ethics of each scenario turn out to be sensitive to details of the story that may seem immaterial to the abstract dilemma.  The question of formulating a general principle that can account for the differing judgments arising in different variants of the story was raised in 1967 as part of an analysis of [[Abortion debate|debates on abortion]] and the [[Principle of double effect|doctrine of double effect]] by the English philosopher [[Philippa Foot]].<ref name="Philippa Foot 1978">Philippa Foot, "[https://www2.econ.iastate.edu/classes/econ362/hallam/Readings/FootDoubleEffect.pdf The Problem of Abortion and the Doctrine of the Double Effect]" in ''Virtues and Vices'' (Oxford: Basil Blackwell, 1978) (originally appeared in the ''Oxford Review'', Number 5, 1967.)</ref>  Later dubbed "the trolley problem" by [[Judith Jarvis Thomson]] in a 1976 article that catalyzed a large literature, the subject refers to the meta-problem of why different judgements are arrived at in particular instances.

Philosophers Judith Thomson,<ref>Judith Jarvis Thomson, ''[https://learning.hccs.edu/faculty/david.poston/phil1301.80361/readings-for-march-31/JJ%20Thomson%20-%20Killing-%20Letting%20Die-%20and%20the%20Trolley%20Problem.pdf Killing, Letting Die, and the Trolley Problem]'', 59 The Monist 204-17 (1976)</ref><ref name="Judith Jarvis Thomson 1985">{{cite journal | last1 = Jarvis Thomson | first1 = Judith | year = 1985 | title = The Trolley Problem | url = http://www.psy.vanderbilt.edu/courses/hon182/thomsontrolley.pdf | journal = Yale Law Journal | volume = 94 | issue = 6| pages = 1395–1415 | doi = 10.2307/796133 | jstor = 796133 }}</ref> [[Frances Kamm]],<ref>{{cite journal | last1 = Myrna Kamm | first1 = Francis | year = 1989 | title = Harming Some to Save Others | journal = Philosophical Studies | volume = 57 | issue = 3| pages = 227–60 | doi = 10.1007/bf00372696 | s2cid = 171045532 }}</ref> and [[Peter Unger]] have also analysed the [[dilemma]] extensively.<ref>Peter Unger, ''[[Living High and Letting Die]]'' (Oxford: Oxford University Press, 1996)</ref> Thomson's 1976 article initiated the literature on the trolley problem as a subject in its own right.  Characteristic of this literature are colorful and increasingly absurd alternative scenarios in which the sacrificed person is instead pushed onto the tracks as a way to stop the trolley, has his organs harvested to save transplant patients, or is killed in more indirect ways that complicate the chain of [[Causality|causation]] and responsibility.

Earlier forms of individual trolley scenarios antedated Foot's publication. [[Frank Chapman Sharp]] included a version in a moral questionnaire given to undergraduates at the [[University of Wisconsin]] in 1905. In this variation, the railway's [[switchman]] controlled the switch, and the lone individual to be sacrificed (or not) was the switchman's child.<ref>Frank Chapman Sharp, ''A Study of the Influence of Custom on the Moral Judgment'' Bulletin of the University of Wisconsin no.236 (Madison, June 1908), 138.</ref><ref>Frank Chapman Sharp, ''Ethics'' (New York: The Century Co, 1928), 42–44, 122.</ref>  German philosopher of law [[Karl Engisch]] discussed a similar dilemma in his [[habilitation thesis]] in 1930, as did German legal scholar Hans Welzel in a work from 1951.<ref>{{Cite book|last=Engisch|first=Karl|title=Untersuchungen über Vorsatz und Fahrlässigkeit im Strafrecht|publisher=O. Liebermann|year=1930|location=Berlin|pages=288}}</ref><ref>Hans Welzel, ''ZStW Zeitschrift für die gesamte Strafrechtswissenschaft'' 63 [1951], 47ff. About the German discussion see also Schuster, Crim Law Forum 34, 237–270 (2023). https://doi.org/10.1007/s10609-023-09452-0</ref> In his commentary on the [[Talmud]], published long before his death in 1953, [[Avrohom Yeshaya Karelitz]] considered the question of whether it is ethical to deflect a projectile from a larger crowd toward a smaller one.<ref>Hazon Ish, HM, Sanhedrin #25, s.v. "veyesh leayen". Available online, http://hebrewbooks.org/14332, page 404</ref> Similarly, in [[The Strike (Westinghouse Studio One)|''The Strike'']], a television play broadcast in the United States on June 7, 1954, a commander in the [[Korean War]] must choose between ordering an air strike on an encroaching enemy force at the cost of his own 20-man patrol unit, or calling off the strike and risking the lives of the main army made up of 500 men.<ref>[https://www.paleycenter.org/collection/item/?q=westinghouse+studio+one&p=4&item=T83:0226 "Studio One: The Strike(TV)"] The Paley Center. Retrieved August 07, 2022.</ref>

Beginning in 2001, the trolley problem and its variants have been used in empirical research on [[moral psychology]]. It has been a topic of popular books.<ref>{{Cite news | url=https://www.nytimes.com/2013/11/24/books/review/would-you-kill-the-fat-man-and-the-trolley-problem.html | title=Clang Went the Trolley| newspaper=The New York Times| date=2013-11-22| last1=Bakewell| first1=Sarah}}</ref> Trolley-style scenarios also arise in discussing the ethics of [[autonomous vehicle]] design, which may require programming to choose whom or what to strike when a [[traffic collision|collision]] appears to be unavoidable.<ref name=":0">{{Cite journal|last1=Lim|first1=Hazel Si Min|last2=Taeihagh|first2=Araz|date=2019|title=Algorithmic Decision-Making in AVs: Understanding Ethical and Technical Concerns for Smart Cities|journal=Sustainability|language=en|volume=11|issue=20|pages=5791|doi=10.3390/su11205791|arxiv=1910.13122|doi-access=free}}</ref>

== Original dilemma ==
Foot's version of the thought experiment, now known as "Trolley Driver", ran as follows:

{{blockquote|Suppose that a judge or magistrate is faced with rioters demanding that a culprit be found for a certain crime and threatening otherwise to take their own bloody revenge on a particular section of the community. The real culprit being unknown, the judge sees himself as able to prevent the bloodshed only by framing some innocent person and having him executed. Beside this example is placed another in which a pilot whose airplane is about to crash is deciding whether to steer from a more to a less inhabited area. To make the parallel as close as possible, it may rather be supposed that he is the driver of a runaway tram, which he can only steer from one narrow track on to another; five men are working on one track and one man on the other; anyone on the track he enters is bound to be killed. In the case of the riots, the mob have five hostages, so that in both examples, the exchange is supposed to be one man's life for the lives of five.<ref name="Philippa Foot 1978" />}}

A [[Utilitarianism|utilitarian]] view asserts that it is obligatory to steer to the track with one man on it. According to classical utilitarianism, such a decision would be not only permissible, but, morally speaking, the better option (the other option being no action at all).<ref>Barcalow, Emmett, ''Moral Philosophy: Theories and Issues.'' Belmont, CA: Wadsworth, 2007. Print.</ref> This fact makes diverting the trolley obligatory. An alternative viewpoint is that since moral wrongs are already in place in the situation, moving to another track constitutes a participation in the moral wrong, making one partially responsible for the death when otherwise no one would be responsible. An opponent of action may also point to the [[commensurability (ethics)|incommensurability]] of human lives. Under some interpretations of [[duty|moral obligation]], simply being present in this situation and being able to influence its outcome constitutes an obligation to participate. If this is the case, then doing nothing would be considered an immoral act.

== Empirical research ==
{{See also|Moral psychology|Dual process theory (moral psychology)}}
In 2001, [[Joshua Greene (psychologist)|Joshua Greene]] and colleagues published the results of the first significant empirical investigation of people's responses to trolley problems.<ref>{{Cite journal|last1=Greene|first1=Joshua D.|last2=Sommerville|first2=R. Brian|last3=Nystrom|first3=Leigh E.|last4=Darley|first4=John M.|last5=Cohen|first5=Jonathan D.|date=2001-09-14|title=An fMRI Investigation of Emotional Engagement in Moral Judgment|journal=Science|language=en|volume=293|issue=5537|pages=2105–2108|doi=10.1126/science.1062872|issn=0036-8075|pmid=11557895|bibcode=2001Sci...293.2105G|s2cid=1437941}}</ref> Using [[functional magnetic resonance imaging]], they demonstrated that "personal" dilemmas (like pushing a man off a footbridge) preferentially engage brain regions associated with emotion, whereas "impersonal" dilemmas (like diverting the trolley by flipping a switch) preferentially engaged regions associated with controlled reasoning. On these grounds, they advocate for the [[Dual process theory (moral psychology)|dual-process account of moral decision-making]]. Since then, numerous other studies have employed trolley problems to study moral judgment, investigating topics like the role and influence of stress,<ref>{{Cite journal|last1=Youssef|first1=Farid F.|last2=Dookeeram|first2=Karine|last3=Basdeo|first3=Vasant|last4=Francis|first4=Emmanuel|last5=Doman|first5=Mekaeel|last6=Mamed|first6=Danielle|last7=Maloo|first7=Stefan|last8=Degannes|first8=Joel|last9=Dobo|first9=Linda|title=Stress alters personal moral decision making|journal=Psychoneuroendocrinology|volume=37|issue=4|pages=491–498|doi=10.1016/j.psyneuen.2011.07.017|pmid=21899956|year=2012|s2cid=30489504}}</ref> emotional state,<ref>{{Cite journal|last1=Valdesolo|first1=Piercarlo|last2=DeSteno|first2=David|date=2006-06-01|title=Manipulations of Emotional Context Shape Moral Judgment|journal=Psychological Science|language=en|volume=17|issue=6|pages=476–477|doi=10.1111/j.1467-9280.2006.01731.x|issn=0956-7976|pmid=16771796|s2cid=13511311}}</ref> impression management,<ref>{{Cite journal|last=Rom, Sarah C.|first=Paul, Conway|date=2017-08-30|title=The strategic moral self:self-presentation shapes moral dilemma judgments|journal=Journal of Experimental Social Psychology|language=en|volume=74|pages=24–37|doi=10.1016/j.jesp.2017.08.003|s2cid=52205265 |issn=0022-1031|url=http://purl.flvc.org/fsu/fd/FSU_libsubv1_wos_000418989800003}}</ref> levels of anonymity,<ref>{{Cite journal|first1=Minwoo|last1=Lee|first2=Sunhae|last2=Sul|first3=Hackjin|last3=Kim|date=2018-06-18|title=Social observation increases deontological judgments in moral dilemmas|journal=Evolution and Human Behavior|volume=39|issue=6|pages=611–621|doi=10.1016/j.evolhumbehav.2018.06.004|s2cid=150247068|issn=1090-5138}}</ref> different types of brain damage,<ref>{{Cite journal|last1=Ciaramelli|first1=Elisa|last2=Muccioli|first2=Michela|last3=Làdavas|first3=Elisabetta|last4=Pellegrino|first4=Giuseppe di|date=2007-06-01|title=Selective deficit in personal moral judgment following damage to ventromedial prefrontal cortex|journal=Social Cognitive and Affective Neuroscience|language=en|volume=2|issue=2|pages=84–92|doi=10.1093/scan/nsm001|issn=1749-5024|pmc=2555449|pmid=18985127}}</ref> physiological arousal,<ref>{{Cite journal|last1=Navarrete|first1=C. David|last2=McDonald|first2=Melissa M.|last3=Mott|first3=Michael L.|last4=Asher|first4=Benjamin|date=2012-04-01|title=Virtual morality: Emotion and action in a simulated three-dimensional "trolley problem".|journal=Emotion|language=en|volume=12|issue=2|pages=364–370|doi=10.1037/a0025561|pmid=22103331|s2cid=34621870 |issn=1931-1516}}</ref> different neurotransmitters,<ref>{{Cite journal|last1=Crockett|first1=Molly J.|last2=Clark|first2=Luke|last3=Hauser|first3=Marc D.|last4=Robbins|first4=Trevor W.|date=2010-10-05|title=Serotonin selectively influences moral judgment and behavior through effects on harm aversion|journal=Proceedings of the National Academy of Sciences|language=en|volume=107|issue=40|pages=17433–17438|doi=10.1073/pnas.1009396107|issn=0027-8424|pmc=2951447|pmid=20876101|bibcode=2010PNAS..10717433C|doi-access=free}}</ref> and genetic factors<ref>{{Cite journal|last1=Bernhard|first1=Regan M.|last2=Chaponis|first2=Jonathan|last3=Siburian|first3=Richie|last4=Gallagher|first4=Patience|last5=Ransohoff|first5=Katherine|last6=Wikler|first6=Daniel|last7=Perlis|first7=Roy H.|last8=Greene|first8=Joshua D.|date=2016-12-01|title=Variation in the oxytocin receptor gene (OXTR) is associated with differences in moral judgment|journal=Social Cognitive and Affective Neuroscience|language=en|volume=11|issue=12|pages=1872–1881|doi=10.1093/scan/nsw103|issn=1749-5016|pmc=5141955|pmid=27497314}}</ref> on responses to trolley dilemmas.

Trolley problems have been used as a measure of utilitarianism, but their usefulness for such purposes has been widely [[#Criticism|criticized]].<ref name="Bauman2014" /><ref name="Kahane2018" /><ref name="Kahane2015">{{cite journal |last1=Kahane |first1=Guy |title=Sidetracked by trolleys: Why sacrificial moral dilemmas tell us little (or nothing) about utilitarian judgment |journal=Social Neuroscience |date=20 March 2015 |volume=10 |issue=5 |pages=551–560 |doi=10.1080/17470919.2015.1023400|pmid=25791902 |pmc=4642180 |doi-access=free }}</ref>

In 2017, a group led by [[Michael Stevens (educator)|Michael Stevens]] performed the first realistic trolley-problem experiment, where subjects were placed alone in what they thought was a train-switching station, and shown footage that they thought was real (but was actually prerecorded) of a train going down a track, with five workers on the main track, and one on the secondary track; the participants had the option to pull the lever to divert the train toward the secondary track. Five of the seven participants did not pull the lever.<ref name="VSauce">{{cite episode|url=https://www.youtube.com/watch?v=1sl5KJ69qiA |archive-url=https://ghostarchive.org/varchive/youtube/20211212/1sl5KJ69qiA| archive-date=2021-12-12 |url-status=live|title=The Greater Good|series=Mind Field|season=2|number=1|last=Stevens|first=Michael |author-link=Michael Stevens (educator)|date=6 December 2017|access-date=23 December 2018}}{{cbignore}}</ref>{{Better source needed|date=July 2022}}

=== Survey data ===
The trolley problem has been the subject of many surveys in which about 90% of respondents have chosen to kill the one and save the five.<ref>{{cite magazine|url=http://healthland.time.com/2011/12/05/would-you-kill-one-person-to-save-five-new-research-on-a-classic-debate/|title='Trolley Problem': Virtual-Reality Test for Moral Dilemma – TIME.com|magazine=TIME.com}}</ref> If the situation is modified where the one sacrificed for the five was a relative or romantic partner, respondents are much less likely to be willing to sacrifice the one life.<ref>{{cite journal |author=April Bleske-Rechek |author2=Lyndsay A. Nelson |author3=Jonathan P. Baker |author4=Mark W. Remiker |author5=Sarah J. Brandt|title=Evolution and the trolley problem: People save five over one unless the one is young, genetically related, or a romantic partner |url=http://137.140.1.71/jsec/articles/volume4/issue3/bleskefinal.pdf |journal=Journal of Social, Evolutionary, and Cultural Psychology |archive-url=https://web.archive.org/web/20120411182922/http://137.140.1.71/jsec/articles/volume4/issue3/bleskefinal.pdf |volume=4 |issue=3 |pages=115–127 |year=2010|doi=10.1037/h0099295 |archive-date=2012-04-11 }}</ref>

A 2009 survey by David Bourget and [[David Chalmers]] shows that 68% of professional philosophers would switch (sacrifice the one individual to save five lives) in the case of the trolley problem,  8% would not switch, and the remaining 24% had another view or could not answer.<ref>{{cite web|last1=Bourget|first1=David|last2=Chalmers|first2=David J.|year=2013|title=What do Philosophers believe?|url=http://philpapers.org/archive/BOUWDP|access-date=11 May 2013}}</ref>

=== Criticism ===
In a 2014 paper published in the ''Social and Personality Psychology Compass'',<ref name="Bauman2014">{{cite journal|last1=Bauman|first1=Christopher W.|last2=McGraw|first2=A. Peter|last3=Bartels|first3=Daniel M.|last4=Warren|first4=Caleb|title=Revisiting External Validity: Concerns about Trolley Problems and Other Sacrificial Dilemmas in Moral Psychology|date=September 4, 2014|journal=Social and Personality Psychology Compass|volume = 8|issue=9|pages=536–554|doi=10.1111/spc3.12131|s2cid=11170070 |url=http://www.escholarship.org/uc/item/5j0215cr}}</ref> researchers criticized the use of the trolley problem, arguing, among other things, that the scenario it presents is too extreme and unconnected to real-life moral situations to be useful or educational.<ref>{{cite web|url=https://www.theatlantic.com/health/archive/2014/07/what-if-one-of-the-most-popular-experiments-in-psychology-is-worthless/374931/|title=Is One of the Most Popular Psychology Experiments Worthless?|last=Khazan|first=Olga|date=July 24, 2014|work=[[The Atlantic]]}}</ref>

In her 2017 paper, Nassim JafariNaimi<ref>{{Cite journal|title=Our Bodies in the Trolley's Path, or Why Self-driving Cars Must *Not* Be Programmed to Kill|journal = Science, Technology, & Human Values|volume = 43|issue = 2|pages = 302–323|last=JafariNaimi|first=Nassim|doi=10.1177/0162243917718942|year = 2018|s2cid = 148793137}}</ref> lays out the reductive nature of the trolley problem in framing ethical problems that serves to uphold an impoverished version of utilitarianism. She argues that the popular argument that the trolley problem can serve as a template for algorithmic morality is based on fundamentally flawed premises that serve the most powerful with potentially dire consequences on the future of cities.<ref>{{cite news |title=Why Self-Driving Cars Must Be Programmed to Kill |url=https://www.technologyreview.com/s/542626/why-self-driving-cars-must-be-programmed-to-kill/ |work=MIT Technology Review |date=October 22, 2015 |language=en}}</ref>

In 2017, in his book ''On Human Nature'', [[Roger Scruton]] criticises the usage of ethical dilemmas such as the trolley problem and their usage by philosophers such as [[Derek Parfit]] and [[Peter Singer]] as ways of illustrating their ethical views. Scruton writes, "These 'dilemmas' have the useful character of eliminating from the situation just about every morally relevant relationship and reducing the problem to one of arithmetic alone." Scruton believes that just because one would choose to change the track so that the train hits the one person instead of the five does not mean that they are necessarily a [[consequentialist]]. As a way of showing the flaws in consequentialist responses to ethical problems, Scruton points out paradoxical elements of belief in utilitarianism and similar beliefs. He believes that [[Robert Nozick|Nozick's]] [[experience machine]] thought experiment definitively disproves [[hedonism]].<ref>{{cite book |last=Scruton |first=Roger |author-link=Roger Scruton |title=On Human Nature |publisher=Princeton |edition=1st |date=2017 |pages=79–112 |isbn=978-0-691-18303-9}}</ref> In his 2017 article ''The Trolley Problem and the Dropping of Atomic Bombs,'' Masahiro Morioka considers the dropping of atomic bombs as an example of the trolley problem and points out that there are five "problems of the trolley problem", namely, 1) rarity, 2) inevitability, 3) safety zone, 4) possibility of becoming a victim, and 5) the lack of perspective of the dead victims who were deprived of freedom of choice.<ref>{{Cite web|url=https://www.bbc.com/future/article/20200804-can-nuclear-war-ever-be-morally-justified|title=Can nuclear war be morally justified?|first=Richard|last=Fisher|website=www.bbc.com|accessdate=19 April 2023}}; {{Cite journal|title=The Trolley Problem and the Dropping of Atomic Bombs |journal=Journal of Philosophy of Life |date=2017 |volume=7 |issue=2 |pages=316–337 |url=https://philarchive.org/rec/MORTTP-7 |last1=Morioka |first1=Masahiro }}</ref>

In a 2018 article published in ''Psychological Review'', researchers pointed out that, as measures of utilitarian decisions, sacrificial dilemmas such as the trolley problem measure only one facet of proto-utilitarian tendencies, namely permissive attitudes toward instrumental harm, while ignoring impartial concern for the greater good. As such, the authors argued that the trolley problem provides only a partial measure of utilitarianism.<ref name="Kahane2018">{{cite journal |last1=Kahane |first1=Guy |last2=Everett |first2=Jim A. C. |last3=Earp |first3=Brian D. |last4=Caviola |first4=Lucius |last5=Faber |first5=Nadira S. |last6=Crockett |first6=Molly J. |last7=Savulescu |first7=Julian |title=Beyond sacrificial harm: A two-dimensional model of utilitarian psychology. |journal=Psychological Review |date=March 2018 |volume=125 |issue=2 |pages=131–164 |doi=10.1037/rev0000093|pmid=29265854 |pmc=5900580 }}</ref>

== Related problems ==
[[File:The trolley problem.svg|thumbnail|upright=1.5|Five cases of the trolley problem: the original Switch, the Fat Man, the Fat Villain, the Loop, and the Man in the Yard]]
Trolley problems highlight the difference between [[deontological ethics|deontological]] and [[consequentialism|consequentialist]] ethical systems. The central question that these dilemmas bring to light is on whether or not it is right to actively inhibit the [[utility]] of an individual if doing so produces a greater utility for other individuals.<ref name=":0" />

The basic Switch form of the trolley problem also supports comparison to other, related dilemmas:

=== The Fat Man ===
{{blockquote|As before, a trolley is hurtling down a track towards five people. You are on a bridge under which it will pass, and you can stop it by putting something very heavy in front of it. As it happens, there is a fat man next to you – your only way to stop the trolley is to push him over the bridge and onto the track, killing him to save five. Should you proceed?}}

Resistance to this course of action seems strong; when asked, a majority of people will approve of pulling the switch to save a net of four lives, but will disapprove of pushing the fat man to save a net of four lives.<ref>Peter Singer, ''Ethics and Intuitions'' The Journal of Ethics (2005). http://www.utilitarian.net/singer/by/200510--.pdf {{Webarchive|url=https://web.archive.org/web/20160617002439/http://www.utilitarian.net/singer/by/200510--.pdf |date=2016-06-17 }}</ref> This has led to attempts to find a relevant moral distinction between the two cases.

One possible distinction could be that in the first case, one does not intend harm towards anyone – harming the one is just a [[Unintended consequence|side effect]] of switching the trolley away from the five. However, in the second case, harming the one is an integral part of the plan to save the five. This solution is essentially an application of the [[doctrine of double effect]], which says that one may take action that has bad side effects, but deliberately intending harm (even for good causes) is wrong. So, the action is permissible even if the harm to the innocent person is foreseen, so long as it is not intended. This is an argument which [[Shelly Kagan]] considers (and ultimately rejects) in his first book ''The Limits of Morality''.<ref>Shelly Kagan, ''The Limits of Morality'' (Oxford: Oxford University Press, 1989)</ref>

== Implications for autonomous vehicles ==
Variants of the original Trolley Driver dilemma arise in the design of software to control [[autonomous car]]s.<ref name=":0" /> Situations are anticipated where a potentially fatal collision appears to be unavoidable, but in which choices made by the car's [[software]], such as into whom or what to crash, can affect the particulars of the deadly outcome. For example, should the software value the safety of the car's occupants more, or less, than that of potential victims outside the car.<ref>{{cite magazine|url=https://www.theatlantic.com/technology/archive/2013/10/the-ethics-of-autonomous-cars/280360/|title=The Ethics of Autonomous Cars|date=October 8, 2013|magazine=The Atlantic|author=Patrick Lin}}</ref><ref>{{cite journal|arxiv=1510.03346|author2=Azim Shariff|title=Autonomous Vehicles Need Experimental Ethics: Are We Ready for Utilitarian Cars?|date=October 13, 2015|author1=Jean-François Bonnefon|author3=Iyad Rahwan|doi=10.1126/science.aaf2654|pmid=27339987|volume=352|issue=6293|journal=Science|pages=1573–1576|bibcode=2016Sci...352.1573B|s2cid=35400794}}</ref><ref>{{cite web|url=http://www.technologyreview.com/view/542626/why-self-driving-cars-must-be-programmed-to-kill/|title=Why Self-Driving Cars Must Be Programmed to Kill|date=October 22, 2015|publisher=MIT Technology review|author=Emerging Technology From the arXiv|access-date=October 24, 2015|archive-date=January 26, 2016|archive-url=https://web.archive.org/web/20160126085929/http://www.technologyreview.com/view/542626/why-self-driving-cars-must-be-programmed-to-kill/|url-status=dead}}</ref><ref>{{Cite journal|last2=Shariff|first2=Azim|last3=Rahwan|first3=Iyad|year=2016|title=The social dilemma of autonomous vehicles|journal=Science|volume=352|issue=6293|pages=1573–1576|doi=10.1126/science.aaf2654|first1=Jean-François|last1=Bonnefon|pmid=27339987|arxiv=1510.03346|bibcode=2016Sci...352.1573B|s2cid=35400794}}</ref>

A platform called [[Moral Machine]]<ref name="Moral Machine">{{Cite web|url=http://moralmachine.mit.edu/|title=Moral Machine|website=Moral Machine|access-date=2019-01-31}}</ref> was created by [[MIT Media Lab]] to allow the public to express their opinions on what decisions autonomous vehicles should make in scenarios that use the trolley problem paradigm. Analysis of the data collected through Moral Machine showed broad differences in relative preferences among different countries.<ref name="Awad2018">{{cite journal |last1=Awad |first1=Edmond |last2=Dsouza |first2=Sohan |last3=Kim |first3=Richard |last4=Schulz |first4=Jonathan |last5=Henrich |first5=Joseph |last6=Shariff |first6=Azim |last7=Bonnefon |first7=Jean-François |last8=Rahwan |first8=Iyad |title=The Moral Machine experiment |journal=Nature |date=October 24, 2018 |volume=563 |issue=7729 |pages=59–64 |doi=10.1038/s41586-018-0637-6 |pmid=30356211 |hdl=10871/39187 |hdl-access=free |bibcode=2018Natur.563...59A |s2cid=53029241 }}</ref> Other approaches make use of virtual reality to assess human behavior in experimental settings.<ref>{{cite journal |last1=Sütfeld |first1=Leon R. |last2=Gast |first2=Richard |last3=König | first3=Peter | last4=Pipa | first4=Gordon |title=Using Virtual Reality to Assess Ethical Decisions in Road Traffic Scenarios: Applicability of Value-of-Life-Based Models and Influences of Time Pressure |journal=Frontiers in Behavioral Neuroscience |volume= 11|pages= 122|doi=10.3389/fnbeh.2017.00122|pmid=28725188 |pmc=5496958 |year=2017 |doi-access=free }}</ref><ref>{{cite journal |last1=Skulmowski |first1=Alexander |last2=Bunge |first2=Andreas |last3=Kaspar | first3=Kai | last4=Pipa | first4=Gordon |date=December 16, 2014 |title=Forced-choice decision-making in modified trolley dilemma situations: a virtual reality and eye tracking study |journal=Frontiers in Behavioral Neuroscience |volume= 8|pages= 426|doi= 10.3389/fnbeh.2014.00426 |pmid=25565997 |pmc=4267265 |doi-access=free }}</ref><ref>{{Cite journal|last1=Francis|first1=Kathryn B.|last2=Howard|first2=Charles|last3=Howard|first3=Ian S.|last4=Gummerum|first4=Michaela|last5=Ganis|first5=Giorgio|last6=Anderson|first6=Grace|last7=Terbeck|first7=Sylvia|date=October 10, 2016|title=Virtual Morality: Transitioning from Moral Judgment to Moral Action?|journal=PLOS ONE|volume=11|issue=10|pages=e0164374|doi=10.1371/journal.pone.0164374|pmid=27723826|issn=1932-6203|bibcode=2016PLoSO..1164374F|pmc=5056714|doi-access=free}}</ref><ref>{{Cite journal|last1=Patil|first1=Indrajeet|last2=Cogoni|first2=Carlotta|last3=Zangrando|first3=Nicola|last4=Chittaro|first4=Luca|last5=Silani|first5=Giorgia|date=January 2, 2014|title=Affective basis of judgment-behavior discrepancy in virtual experiences of moral dilemmas|journal=Social Neuroscience|volume=9|issue=1|pages=94–107|doi=10.1080/17470919.2013.870091|issn=1747-0919|pmid=24359489|s2cid=706534|url=http://psyarxiv.com/ry3ap/}}</ref> However, some argue that the investigation of trolley-type cases is not necessary to address the ethical problem of driverless cars, because the trolley cases have a serious practical limitation. It would need to be top-down plan in order to fit the current approaches of addressing emergencies in [[artificial intelligence]].<ref>{{Cite journal|last=Himmelreich|first=Johannes|date=June 1, 2018|title=Never Mind the Trolley: The Ethics of Autonomous Vehicles in Mundane Situations|journal=Ethical Theory and Moral Practice|language=en|volume=21|issue=3|pages=669–684|doi=10.1007/s10677-018-9896-4|s2cid=150184601|issn=1572-8447}}</ref>

Also, a question remains of whether the law should dictate the ethical standards that all autonomous vehicles must use, or whether individual autonomous car owners or drivers should determine their car's ethical values, such as favoring safety of the owner or the owner's family over the safety of others.<ref name=":0" /> Although most people would not be willing to use an automated car that might sacrifice themselves in a life-or-death dilemma, some{{Who|date = August 2019}} believe the somewhat counterintuitive claim that using mandatory ethics values would nevertheless be in their best interest. According to Gogoll and Müller, "the reason is, simply put, that [personalized ethics settings]<!-- original text: "a PES regime" --> would most likely result in a [[prisoner’s dilemma]]."<ref>{{Cite journal|last1=Gogoll|first1=Jan|last2=Müller|first2=Julian F.|date=June 1, 2017|title=Autonomous Cars: In Favor of a Mandatory Ethics Setting|journal=Science and Engineering Ethics|language=en|volume=23|issue=3|pages=681–700|doi=10.1007/s11948-016-9806-x|pmid=27417644|s2cid=3632738|issn=1471-5546}}</ref>

In 2016, the German government appointed a commission to study the ethical implications of autonomous driving.<ref>{{cite web|url=http://www.bmvi.de/SharedDocs/DE/Publikationen/G/bericht-der-ethik-kommission.html|title=Bericht der Ethik-Kommission Automatisiertes und vernetztes Fahren|author=BMVI Commission|date=June 20, 2016|publisher=Federal Ministry of Transport and Digital Infrastructure (German: Bundesministerium für Verkehr und digitale Infrastruktur)|archive-url=https://web.archive.org/web/20171115224017/http://www.bmvi.de/SharedDocs/DE/Publikationen/G/bericht-der-ethik-kommission.html|archive-date=November 15, 2017|url-status=dead}}</ref><ref name=BMVI-rules>{{cite web|url=https://www.bmvi.de/SharedDocs/EN/publications/report-ethics-commission.html|title=Ethics Commission's complete report on automated and connected driving|author=BMVI Commission|date=28 August 2017|publisher=Federal Ministry of Transport and Digital Infrastructure (German: Bundesministerium für Verkehr und digitale Infrastruktur)|access-date=January 20, 2021|archive-url=https://web.archive.org/web/20170915110611/http://www.bmvi.de/SharedDocs/EN/publications/report-ethics-commission.html|archive-date=15 September 2017}}</ref> The commission adopted 20 rules to be implemented in the laws that will govern the ethical choices that autonomous vehicles will make.<ref name=BMVI-rules />{{rp|10–13}} Relevant to the trolley dilemma is this rule:
<blockquote>
8. Genuine dilemmatic decisions, such as a decision between one human life and another, depend on the actual specific situation, incorporating “unpredictable” behaviour by parties affected. They can thus not be clearly standardized, nor can they be programmed such that they are ethically unquestionable. Technological systems must be designed to avoid accidents. However, they cannot be standardized to a complex or intuitive assessment of the impacts of an accident in such a way that they can replace or anticipate the decision of a responsible driver with the moral capacity to make correct judgements. It is true that a human driver would be acting unlawfully if he killed a person in an emergency to save the lives of one or more other persons, but he would not necessarily be acting culpably. Such legal judgements, made in retrospect and taking special circumstances into account, cannot readily be transformed into abstract/general ex ante appraisals and thus also not into corresponding programming activities. …<ref name=BMVI-rules />{{rp|11}}
</blockquote>

== See also ==
{{div col|colwidth=20em}}
* [[Lesser of two evils principle]]
* [[Lifeboat ethics]]
* [[Omission bias]]
* ''[[R. v. Dudley and Stephens]]''
* {{section link|The Case of the Speluncean Explorers|Similar real cases}}
* [[Tunnel problem]]
* [[Violinist (thought experiment)]]
* [[Virtue ethics]]
{{div col end}}

== References ==
{{Reflist|30em}}

== External links ==
{{Wikiquote}}
{{Spoken Wikipedia|Trolley Problem.ogg|date=2012-04-30}}
* [http://journal.frontiersin.org/Journal/10.3389/fnbeh.2014.00426/full Forced-choice decision-making in modified trolley dilemma situations: a virtual reality and eye tracking study]

[[Category:1967 introductions]]
[[Category:Dilemmas]]
[[Category:Metaphors referring to objects]]
[[Category:Moral psychology]]
[[Category:Philosophical problems]]
[[Category:Thought experiments in ethics]]
[[Category:Virtue ethics]]