Machine-learning systems are increasingly worming their way through our everyday lives, challenging our moral and social values and the rules that govern them. These days, virtual assistants threaten the privacy of the home; news recommenders shape the way we understand the world; risk-prediction systems tip social workers on which children to protect from abuse; while data-driven hiring tools also rank your chances of landing a job.
However, the ethics of machine learning remains blurry for many.
Searching for articles on the subject for the young engineers attending the Ethics and Information and Communications Technology course at UCLouvain, Belgium, I was particularly struck by the case of Joshua Barbeau, a 33-year-old man who used a website called Project December to create a conversational robot – a chatbot – that would simulate conversation with his deceased fiancée, Jessica.
Known as a deadbot, this type of chatbot allowed Barbeau to exchange text messages with an artificial “Jessica”. Despite the ethically controversial nature of the case, I rarely found materials that went beyond the mere factual aspect and analysed the case through an explicit normative lens: why would it be right or wrong, ethically desirable or reprehensible, to develop a deadbot?
Before we grapple with these questions, let’s put things into context: Project December was created by the games developer Jason Rohrer to enable people to customise chatbots with the personality they wanted to interact with, provided that they paid for it. The project was built drawing on an API of GPT-3, a text-generating language model by the artificial intelligence research company OpenAI. Barbeau’s case opened a rift between Rohrer and OpenAI because the company’s guidelines explicitly forbid GPT-3 to be used for sexual, amorous, self-harm or bullying purposes.
Calling OpenAI’s position as hyper-moralistic and arguing that people like Barbeau were “consenting adults”, Rohrer shut down the GPT-3 version of Project December.
While we may all have intuitions about whether it is right or wrong to develop a machine-learning deadbot, spelling out its implications hardly makes for an easy task. This is why it is important to address the ethical questions raised by the case, step by step.
Is Barbeau’s consent enough to develop Jessica’s deadbot?
Since Jessica was a real (albeit dead) person, Barbeau consenting to the creation of a deadbot mimicking her seems insufficient. Even when they die, people are not mere things with which others can do as they please. This is why our societies consider it wrong to desecrate or to be disrespectful to the memory of the dead. In other words, we have certain moral obligations towards the dead, insofar as death does not necessarily imply that people cease to exist in a morally relevant way.
Likewise, the debate is open as to whether we should protect the dead’s fundamental rights (e.g., privacy and personal data). Developing a deadbot replicating someone’s personality requires great amounts of personal information such as social network data (see what Microsoft or Eternime propose) which have proven to reveal highly sensitive traits.
If we agree that it is unethical to use people’s data without their consent while they are alive, why should it be ethical to do so after their death? In that sense, when developing a deadbot, it seems reasonable to request the consent of the one whose personality is mirrored – in this case, Jessica.
When the imitated person gives the green light
Thus, the second question is: would Jessica’s consent be enough to consider her deadbot’s creation ethical? What if it was degrading to her memory?
The limits of consent are, indeed, a controversial issue. Take as a paradigmatic example the “Rotenburg Cannibal”, who was sentenced to life imprisonment despite the fact that his victim had agreed to be eaten. In this regard, it has been argued that it is unethical to consent to things that can be detrimental to ourselves, be it physically (to sell one’s own vital organs) or abstractly (to alienate one’s own rights).
In what specific terms something might be detrimental to the dead is a particularly complex issue that I will not analyse in full. It is worth noting, however, that even if the dead cannot be harmed or offended in the same way than the living, this does not mean that they are invulnerable to bad actions, nor that these are ethical. The dead can suffer damages to their honour, reputation or dignity (for example, posthumous smear campaigns), and disrespect toward the dead also harms those close to them. Moreover, behaving badly toward the dead leads us to a society that is more unjust and less respectful with people’s dignity overall.
Finally, given the malleability and unpredictability of machine-learning systems, there is a risk that the consent provided by the person mimicked (while alive) does not mean much more than a blank check on its potential paths.
Taking all of this into account, it seems reasonable to conclude if the deadbot’s development or use fails to correspond to what the imitated person has agreed to, their consent should be considered invalid. Moreover, if it clearly and intentionally harms their dignity, even their consent should not be enough to consider it ethical.
Who takes responsibility?
A third issue is whether artificial intelligence systems should aspire to mimic any kind of human behaviour (irrespective here of whether this is possible).
This has been a long-standing concern in the field of AI and it is closely linked to the dispute between Rohrer and OpenAI. Should we develop artificial systems capable of, for example, caring for others or making political decisions? It seems that there is something in these skills that make humans different from other animals and from machines. Hence, it is important to note instrumentalising
