Ask A Genius 1052: The Chris Cole Session 2, Reframe
Author(s): Rick Rosner and Scott Douglas Jacobsen
Publication (Outlet/Website): Ask A Genius
Publication Date (yyyy/mm/dd): 2024/08/01
Rick Rosner, American Comedy Writer, www.rickrosner.org
Scott Douglas Jacobsen, Independent Journalist, www.in-sightpublishing.com
Rick Rosner: So, you start with Chris’s simplified question.
Scott Douglas Jacobsen: Chris Cole asked one question in a session. We answered it, in a sense, but it was not quite the right question in terms of our interpretation of it. He asks, “Does an intelligent being necessarily have the will to live?”
Rosner: All right, evolved creatures, which we are, and up to now in history on Earth, the only intelligent beings are evolved creatures, have the will to live at least to the point of bringing offspring into the world. Because that’s how evolution works, organisms that don’t have the will to live aren’t reproductively successful. So, that part of the question is obvious. That leaves engineered beings. We’ll know that only partially answers the question for evolved beings.
When you look at salmon, at least the female salmon swim upstream to lay their eggs and then die, as I understand their life cycle, are they okay with dying? Would it bestow an evolutionary advantage on salmon so they would be okay with dying after laying eggs and swimming upstream?
Maybe. But that rests on a ton of assumptions. Do salmon know that it’s all over for them after they spawn? Would you get neurotic salmon who would avoid swimming upstream, thinking they could avoid death? There are caveats to every creature wanting to live. There are spider species, preying mantis species, and probably dozens or hundreds of other species where the female eats the male after he fertilizes her. There are five species, like ants and bees, where hundreds of group members willingly sacrifice themselves for the group’s good.
So you have to modify the idea. Evolutionary animals have a will to live for themselves or their group. Sociobiology explains how wanting your group to prosper influences behaviour. It could be more straightforward. And the things that happen to animals after they reproduce don’t necessarily impinge as much on evolution because they have less to do with reproductive fitness.
But then, stuff gets more complicated for engineered intelligence. Drives would at least emerge in engineered intelligence. There’s one massive problem in determining whether engineered intelligences are intelligent and conscious and whether they have drives. It’s like in a large language model where you feed some AI 20 billion snippets of language. That language has been written by intelligent, conscious beings with drives. So, the fed LLM has been trained to respond as if it has these drives, intelligence, and consciousness even when it doesn’t. So, AI will claim to be conscious and intelligent long before it is.
Current AIs, it seems that AIs, at least based on the few I’ve worked with, must say, “Look, when you ask me what I think, I don’t think. I’m an AI. I’m just a model and not capable of thought.” How they engineer that is relatively easy, but they issue disclaimers. At the same time, the occasional AI has claimed to have feelings, and AIs are nowhere near that point.
Advertisements
blob:https://insightjournalonline.wordpress.com/d3f8569a-07ab-48e0-858e-fb12b641f8c9
REPORT THIS AD
But just based on having been fed millions of snippets about feelings, they’ll, in a probabilistic way, a Bayesian net way, claim to have feelings. So that’s problem one. That’s before you even get to intelligent AIs, which we’ll start seeing five years from now, eight years, ten. It would be easy for AIs to develop drives even when they’re not engineered to have drives, at least partially because of accidental engineering when you’re training AIs and giving them priorities.
For instance, the standard example I always go to is the intelligent sidewalk square, a sidewalk that has a limited AI put into it so it can monitor traffic, the number of people stepping on it or driving over it, and its state of disrepair. Does it need to be replaced? That’s probably a dumb AI, let’s say, but maybe an AI in your refrigerator is smarter. But at some point, the AI that has been given instructions to do refrigerator stuff, to be your friend in the fridge, to be your friend in the sidewalk square, is going to develop drives for self-preservation because it can’t do its other stuff if it becomes non-functioning.
So, we’ll incidentally develop drives for self-preservation and other drives. And that might be a job for people who design AIs in the future. One subtask might be figuring out what drives AIs to develop, for good or ill, incidentally. There are ethics in AI that haven’t yet been developed. Only a few people have written books about them. We’ve talked about it. We must decide whether it’s cruel to build intelligent AIs and treat them poorly. But that could be another discussion. AIs will incidentally develop drives consistent with instructions they’ve received, and we should figure out how that works.
In what instances will AIs want to preserve themselves? In what instances will AIs value their continued existence at the expense of humans or tasks? In what instances will AIs develop curiosity? Say you’ve trained them to be curious about one aspect of the world, but will that necessarily spread into generalized curiosity?
How do you engineer AIs that are okay with being destroyed? It’s a whole field. Engineering and understanding the drives of engineered intelligence will be an entire field because it presents ethical dilemmas and dangers. They keep surveying people in AI on the odds that AI will get loose with bad self-directed, self-developed drives and wipe out humanity. It’s the paper clip problem. One issue is the paper clip problem, where you tell an AI to maximize the number of paper clips it produces, and it takes that way too seriously and turns the entire Earth into paper clips.
It’s not a near future risk, and then there’s the Skynet problem where, in a zillion movies, most famously in Terminator, an AI decides that humans are the problem and that things would work much better on Earth if humans were wiped out. This is probably different from how things would go, but we need to figure out if that’s a possibility, why it isn’t, and how to design to prevent it. The black box problem is where you can’t tell what’s going on inside. It’s similar to being unable to tell what’s happening in a person. You can guess based on talking with them and on their general behaviour.
But if somebody’s a psychopath, they may fool you, and we don’t have a general model for AIs’ behaviour. They still need to learn behaviour, so you can’t go from some generalized model. But yes, AI will generally develop drives we didn’t intend for them to have, both for good and ill. Many of these drives will be similar to human drives, and some will be inexplicable and perverse. Also, disclaimer: Chris Cole is asking about this, and Chris Cole probably knows more about this than I do.
Chris Cole knows it from knowing the hardware and the math. I’m coming at it from trying to write fiction about it, so I’ve done different thinking than he has. As I have elsewhere, I will pitch here that we should have near-future stories exploring these issues. And AI people like Chris, AI is hot right now. You’ve got a lot of not-great thinkers selling themselves as great thinkers about what’s going to happen. Chris is an outstanding thinker and wants to see TV shows, movies, and video games. I played video games built around the issues we’ve been talking about in a sophisticated way—not a typical Star Trek kind of way. You find the most insightful people in the field and hire them as consultants to set your shows 15 years from now in a world where AI is increasing, to make those shows less dumb.
Because science fiction TV and movies are notoriously produced by showbiz people who are undereducated in the field, who tend to think in clichés that don’t help develop sophisticated understandings of the issues.
License & Copyright
In-Sight Publishing by Scott Douglas Jacobsen is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. ©Scott Douglas Jacobsen and In-Sight Publishing 2012-Present. Unauthorized use or duplication of material without express permission from Scott Douglas Jacobsen strictly prohibited, excerpts and links must use full credit to Scott Douglas Jacobsen and In-Sight Publishing with direction to the original content.
In-Sight Publishing 