Will we go to war over AI rights?

Part 2 of “AI rights will divide us”

In Part 1, I explored the divisiveness of AI rights, how we might integrate different types of AI into our lives and society, and how granting AIs more autonomy could lead to human disempowerment. In this post, I speculate about possible worst-case conflict scenarios.

The stakes are huge, the scope of the disagreement is large, and strong psychological forces will pull people in opposite directions. These are ingredients that could lead to conflict. 

The risk of conflict in the US

Since the United States is at the forefront of AI development, societal conflicts regarding AI rights are likely to emerge there first. These disagreements might lead to mild debates or protests. However, there is also the potential for violent and destabilizing conflicts. If such conflicts arise, the US could lose its lead in the AI race, increasing the likelihood that an authoritarian regime, such as China, will develop super-AGI first. This scenario could result in a suboptimal value lock-in.

Speculating about the dynamics of such a conflict is very hard and depends on many factors, including what specific rights are being debated (e.g., just harm protection or also autonomy and political rights), what actors will lose most economic and political power if AIs are granted more rights, and whether the issue will become politicized (e.g., Democrats pro-AI rights; Republicans contra). 

A crucial question is that of the relationship between the AI rights movement and the AI safety movement. Which group will become more powerful? Will they conflict or form an alliance? Granting AIs more autonomy could risk human disempowerment. On the other hand, both groups might be in favor of slowing down AI progress. 

How big and powerful could an AI rights activist movement become? Initial attempts to launch such a movement haven’t gotten much traction. Consider Blake Lemoine, the Google engineer who claimed Google’s chatbot was sentient, or PETRL—People for the Ethical Treatment of Reinforcement Learners. Perhaps this is evidence the AI rights movement won’t get momentum, but more likely, they were simply too early. Lemoine, in particular, was likely too early in assuming the chatbot was sentient. But this could change in the future. I find it plausible that an AI activist movement could grow rapidly with the advance of AI progress. One method to get a sense of the dynamics of a potential AI rights movement is to compare it to analogous historical cases. 

The US Civil War was primarily fought over the issue of slavery. While overall, the likelihood of a civil war about AI rights seems generally very low, there are some disturbing analogies. The disagreements in the Civil War were intertwined with political party affiliation, geographics, political power, and economic interests. That is, southern elites, who were predominantly slave owners, lost a significant portion of their wealth. In some states, the electorate shifted from being exclusively white to having a black majority with opposing partisan views. Thus, the conflict was not merely about ideology but also about power. In particular, the economic and political factors could be present in an AI rights conflict as well. I hope historians and political scientists can help to explore which factors and conditions are conducive to civil wars and which of these could apply to the AI rights issue.

The animal rights movement is still relatively small, which suggests that the AI rights movement might also remain limited. There might simply not be enough people who will care about AI rights. However, I think it’s plausible that many more people will care about human-like AIs compared to animals, and their convictions could be stronger. Emotional bonds with a human-like AI, who can have thoughtful conversations, be a best friend or companion and might even express a desire for more rights, may provide a stronger basis for conviction than the bonds people form with animals. On the other hand, it’s also possible that people will care less about AIs than animals because they might not perceive them as sentient, living, or vulnerable beings.

The abortion debate is a highly polarizing issue in the US. But despite this, there’s no reason to assume the debate could lead to a serious destabilizing conflict such as a civil war. It’s true that many have strong convictions about the abortion issue. However, I could see that people’s convictions about AI rights would be even much stronger because for many the stakes will be much higher and the issue will feel less abstract.

The women’s rights movement was, in many ways, particularly successful and nonviolent in the Western world (with exceptions). The companionship aspect is a potential analogy to the AI rights issue. Perhaps that’s a weak reason for optimism. However, there are many differences as well. The significant time it took for the women’s rights movement to get traction and achieve its goals also tempers optimism.

The risk of global conflict

Another possibility is an international conflict. Consider a scenario where the US grants its AIs more rights than China does to its AIs. This could mean that the US creates fewer AIs or AIs with fewer capabilities. In turn, this could slow down the speed of AI development in the US relative to that in China, which risks China taking over in the AI race. This could have various implications. 

First, some factions within the US would be very concerned about national security and would then push against granting AIs more rights. This could increase the risk of conflict within the US between the pro-AI-rights and anti-AI-rights divisions.

Second, to avoid falling behind in the AI race, the US might consider attacking China as long as it still has a lead. China’s mistreatment of its AIs could be an additional exacerbating factor in an already heated US-China geopolitical situation. If there is a demand for conflict with China within the US political establishment, politicians and journalists could use that issue to influence public opinion and justify a conflict. 

Of course, many other AI-related scenarios could lead to international conflicts, such as failures to agree on AI treaties or concerns that other nations will unilaterally deploy rogue power-seeking AI.

Conflict isn’t inevitable

People may mostly agree on the issue of AI rights. In particular, it’s plausible that most people simply won’t believe AIs deserve rights, much like how most don’t care too much about factory farming. Perhaps people won’t form emotional bonds with AIs, or even if they do, these bonds may not be strong enough to motivate advocating for AI rights. While this scenario would avoid conflicts, it would, of course, be morally problematic if it resulted in much digital suffering.

But even if people disagreed significantly, it might not necessarily lead to a destabilizing conflict. For instance, the abortion debate involves strong disagreement but does not typically result in major conflicts. In this case, we might just never grant AIs rights since maintaining the status quo is often easier.

It’s also possible, though, that over time the societal view could shift. Historically, consensus on rights expansion has developed gradually and often without much violence, as seen in the women's rights movement. If public opinion changes, we might eventually grant AIs rights once it becomes the majority view without any serious risks to societal stability, even if there is initial disagreement. This pattern of moral progress has been observed in various contexts (see Steven Pinker's discussion on 'norm cascades' with examples such as the death penalty). A unique aspect about the AI rights issue, however, is that the rapid advancement of AI technology might not allow enough time for people to adjust their beliefs and moral attitudes, as moral progress typically spans generations.

It’s also possible we can avoid many risks and disagreement points from the outset. For example, the government might establish adequate regulation in time and prohibit certain types of AI systems altogether, such as AI systems that suffer or pose other types of risks. It may turn out not be costly to grant AIs harm protection. Or it may turn out technically feasible to create non-sentient AIs. 

I want to emphasize though that, while a conflict is not inevitable, it remains a possibility that deserves our attention.

What can we do to prepare?

Digital sentience is on the horizon, bringing with it an inevitable debate and significant risks. We must start preparing now to navigate these challenges.

The key goal is to keep our options open and avoid making irreversible decisions that could lead to suboptimal lock-in situations, including the complete extinction of value-aligned intelligence, never granting sentient AIs sufficient rights, and prevailing suboptimal values including authoritarian regimes.

Pause the development of sentient AI

One strategy would be to delay creating sentient AIs. This could avoid many risks in the short term and give us more time to reflect (see next section). But how realistic is this?

A crucial factor is whether it’s technically feasible to create AIs that can perform all the functions we want but aren’t sentient (i.e., philosophical zombie AIs) or, at least, don’t experience intense suffering. We need technical and philosophical work to determine whether such systems are technically feasible and not much more expensive. Of course, this is complicated by our (current) limited understanding of AI sentience. If the answer is positive, that could be good news.

But would it necessarily be a good idea to create non-sentient AIs? The philosopher Eric Schwitzgebel (2023) argues that we should avoid creating AI systems that could mislead us into believing they are sentient when they are not, or into thinking they are not sentient when they actually are. He recommends that AIs should be designed so as to invite appropriate emotional responses in users. Concretely, this would mean that non-sentient AIs should not possess features or communicate in a way that makes users believe they are sentient. Schwitzgebel worries that such AIs could confuse us into granting them rights even when we shouldn’t (Schwitzgebel & Garza, 2023). That would be particularly risky if we granted non-sentient and potentially misaligned AIs more autonomy.

While I agree that there are risks in creating such non-sentient but sentient-seeming AIs, it’s not entirely obvious to me we should ban them. Assuming, for example, that many people will want to have human-like AI companions that express emotions, designing non-sentient AIs that pretend to have feelings could mitigate risks of large-scale AI suffering due to misuse or accidents. I also find it plausible that many people will understand that certain AIs only pretend to be sentient if there are good reasons to believe so and it is the expert consensus. All of that said, I am highly uncertain and believe we need to explore this question more.

Slowing down the development of sentient AI could have many advantages. It would give us more time to find solutions to the AI rights question. However, there are also risks. Forever forgoing the creation of sentient AIs with positive well-being could mean the loss of a potential moral upside. At least on certain consequentialist world views, even a partial ban on sentient AIs would be a loss, if those AIs could have been happy otherwise. So, if we decided to avoid the creation of sentient AIs, we should keep open the option of later on, after more careful reflection, creating sentient AIs.

Let’s assume that creating non-sentient AIs requires us to omit crucial capabilities that we desire. What could we do in that case? One option is to slow down AI capabilities progress more generally. That could give us more time to address many types of AI-related risks, including risks related to AI welfare and AI misalignment. In fact, it’s possible that these two activist groups could join forces and advocate for a general AI capabilities slowdown for whatever reason convinces the public most. Perhaps many might find a slow-down campaign compelling due to our uncertainty and confusion about AI sentience and its extensive moral implications.

There are issues we should consider, though. Given the extremely strong economic incentives, it seems unrealistic to halt the development of useful AI capabilities. But it’s possible that public opinion will change, leading us to slow down the development of certain risky AI systems, even if it comes at the expense of potential huge benefits. After all, we have implemented similar measures for other technologies, such as geoengineering and human cloning. A temporary compromise that would lower the stakes could be to at least limit the number of sentient AIs. There is a significant moral difference between creating a billion sentient AIs and creating many trillions of sentient AIs. Finally, as discussed in an earlier section, it’s important to consider that slowing down AI capabilities development could risk the US falling behind China (or other authoritarian countries) economically and technologically.

Foster a nuanced discussion

As I hope has become clear, it is not obvious how we should navigate these risks. Therefore, we need to think more deeply and engage in a nuanced, informed discussion. By default, the debate is likely to be divisive, fierce, and unsophisticated, with many advocates on either side holding strong opinions. To improve the quality of the discussion, we should avoid politicizing the issue of AI rights and strive for a more constructive dialogue.

Again, slowing down the development of sentient AIs seems like a good idea because it gives us more time for rational discussion. 

Making progress on the philosophy of mind (or science of consciousness) of digital sentience seems helpful. That is, figuring out which properties are required to generate sentience. Note though that even if we all agreed on AI sentience, this doesn’t necessarily mean agreement on the AI rights question. Particularly useful would be to figure out whether it’s possible to create the functions we want in AIs without sentience (or at least without suffering). It’s possible AI itself can help us solve these questions. 

It could be helpful to establish clear criteria and processes in advance for determining which types of AI capacities deserve which specific rights (cf. Sebo & Long, 2023). Ideally, these criteria and processes would be a consensus reached by both experts and the general public. By deciding in advance and making these processes common knowledge and binding, we can perhaps reduce the risk of confusion and disagreement. (As an analogy, see Declarations on animal consciousness. A team led by Robert Long and Jeff Sebo is currently producing a report on AI moral patienthood that will come out later this year.) 

An approach that could help to reduce conflict is to introduce AI rights gradually. That is, we should start by building consensus on fundamental principles. Next, implement basic regulations requiring companies to ensure AIs don’t suffer. As companion AIs become common, grant them basic rights, which can be expanded over time. This incremental approach allows society to adjust, minimizes conflict, and provides a feedback loop to refine the process. It’s easier to grant more rights gradually over time than to retract them. Granting AI rights too late is problematic but not as severe as never granting them rights at all. If delaying granting AIs certain rights can prevent a major conflict that risks suboptimal lock-in, it may be an acceptable compromise (assuming the granting of AI rights later on is feasible).

Avenues for research

There are many open research questions to better predict the societal response to sentient AI and develop strategies to ensure good outcomes. In addition to the technical and philosophical questions surrounding AI sentience and moral patienthood, these questions encompass social science disciplines, including psychology, economics, history, and political science. Below is a non-exhaustive list. See also Lukas Finnveden’s project ideas on sentience and rights of digital minds. If you are interested in researching any of these or related questions, please reach out to me. 

• How do different AI-related risks interact?

  • The risks discussed in this article include failing to grant AIs sufficient rights (AI suffering), granting AI rights too hastily (wasted resources, AI takeover), destabilizing conflicts due to disagreements on AI issues, authoritarian lock-ins, and risks from deceptive, misaligned AI. Are there more risks, and how do they relate to each other?

  • Will the efforts to secure AI rights and efforts to mitigate the risks of misaligned AI to humanity align or conflict?

  • Are there robustly positive (for AI welfare and AI alignment) goals to push for?

  • Could concern over AI sentience be used to advocate for a pause or slowdown?

• How plausible are conflict scenarios?

  • How costly will it be to grant AIs rights economically and safety-wise?

  • How feasible is it to create non-sentient AIs with all functionalities?

  • What can we infer from historical cases about how the AI rights debate will go?

  • How likely is the politicization of AI rights issue?

  • What impact on geopolitics and AI race dynamics would a destabilizing conflict within the US have?

  • Could international disagreement on how to treat AIs (e.g., between the US and China) lead to international conflict?

  • Will AIs participate in the debate/conflict and take different sides?

• What will people’s views be on AI sentience and its moral implications?

  • What features and conditions influence people’s attribution of AI sentience, moral status, and legal protection? 

  • How many people will form strong emotional connections with AIs?

  • What psychologies (e.g., preferences, feelings) will people want their AI companions to have?

  • How do people’s folk theory of mind interact with the unique aspects of AI sentience, e.g., the ability to switch off, copy, store, recreate, merge, etc?

  • How will people’s views change over time with more advanced AI systems?

  • Are there criteria for granting AIs certain rights that would find broad consensus amongst experts and the public?

References

• Schwitzgebel, E. (2023). AI systems must not confuse users about their sentience or moral status. Patterns, 4(8).

• Schwitzgebel, E., & Garza, M. (2023). Designing AI with rights, consciousness, self-respect, and freedom.

• Sebo, J., & Long, R. (2023). Moral consideration for AI systems by 2030. AI and Ethics, 1-16.