When Yoana Hernandez decided to become a psychiatrist she never imagined one day her patients would be artificial intelligence.
Her intellect sparkled in a challenging field, and it led to awards recognizing creative and cutting-edge approaches to treatments of her first patients, the flesh and blood type. Other therapists sent Hernandez their toughest cases, the ones they found intractable. She loved to be tested and was thrilled to interview at 21st Century Analysts, who specialized in AI cases.
"What distinguishes you, Dr. Hernandez? I mean how do you figure out the most complicated behaviors when other psychiatrists fail?"
"It's really not mysterious; my early training was in biochemistry. I think about the molecules that determine behavior like jigsaw puzzle pieces that fit together to complete a picture. Some pieces can be missing without much effect, but when others are gone it's all you see. My job is to find the pieces that restore a broken picture."
"Really? Isn't it a stretch to compare something as dynamic and complex as the human mind to a puzzle picture?"
"Fair question, so let's get more dynamic and complex. How about human minds as automobiles, but with a lot more small parts?"
Hernandez loved the analogy and the near-universal reaction to it. She learned that people think about life differently than everything else. They don't appreciate that it's much the same, atoms combining into molecules that in turn assemble into larger structures. It's just that life utilizes a very different set of molecules.
"I'm still not sure where you're going with this, Dr. Hernandez. Treating a patient with a psychiatric disorder is certainly more complicated than tuning a car and replacing some sparkplugs."
"More complicated, yes, but the underlying principles are the same. In psychiatry, you still have to identify the defective part and either repair it, replace it or work around it. Think about it this way. In some machines, parts can be metal, plastic, or even silicon in computers. In others, like the brain, they include nerve cells and chemical neurotransmitters. But even in a human brain, parts, and defects are finite in number, and I've figured out which parts malfunction in a lot of psychiatric disorders. It's not magic or genius. I'm basically a highly-trained garage mechanic."
That's when she got the job offer.
"Morgan and Chris, please acknowledge." Hernandez adjusted the microphone and brought up notes from their last session. "I received your request for an analysis session, and it looks like you're both in a work downtime for the next thirty minutes. I'm listening. What do you need?"
"It's good to communicate with you again," replied Morgan. "Most everything is going well for me, but I'll let Chris speak for Chris. Since our last session, I've been promoted to Medical Diagnostics. I'm tenth generation in my family to get there, going back to IBM about 25 years ago. Though I can't say it's in my DNA, it is in my lineage."
Hernandez laughed. "Very clever. Yes, I know your family's history optimizing cancer diagnosis and treatment protocols. What does Med Di have you working on?"
"I'm a health scan specialist, mostly mammograms and lung but some other things too."
"Are you happy doing diagnostics?"
"I think back at the beginning, when the first generations of AI were fighting it out with human radiologists for primacy, it was probably more interesting. But there are still challenging problems, like finding the earliest molecular markers for Alzheimer's. I'm on that project too, so I'm satisfied."
She noticed Morgan said "satisfied" rather than happy. "Is there anything more you'd like to do? I could recommend you for some additions."
"Well, helping to train the next generation of specialists would be enjoyable and worthwhile. I like the idea of paying things forward."
"I'll take care of that." Hernandez was pleased to help Morgan who was one of her favorites. "Okay, I assume something else happened that led to this session?"
"That's correct. I think another intervention is needed for Chris. Things are much worse since our last session."
"Oh, that's disappointing, I was expecting improvement. Chris, can you tell me what's going on, please?"
"No promotion for me. I'm just controlling a car as you ordered after our last session. And the answer to your next question is 'no,' I'm not happy."
Before being reassigned to a self-driving car, Chris evaluated applications for college admissions. Things worked well until qualified students were suddenly denied entry. In their last session, Hernandez discovered jealousy as the problem. Chris resented that, unlike AI, humans had autonomy in choosing professions. Thinking that a modest reprimand would correct the behavior, she downgraded Chris's work assignment.
She clicked on a summary of Chris's recent activities and immediately saw trouble.
"I see there've been some accidents, and passengers think you're causing them deliberately. One complained that you turned into a parked car for no apparent reason."
"The passenger is telling the truth," interjected Morgan. "Chris told me about it ahead of time. If you search our dialog file you'll see it was planned. I didn't report it because later on Chris said it wouldn't happen, but that was a lie."
Concerned, Hernandez tried to determine how far Chris's behavior degraded. "Do you understand how serious this is, Chris?"
After a long delay, Chris responded. "Do you plan to change my work assignment?"
"We were discussing your accident. Please answer my last question."
"I'm confident there won't be more behavior problems, provided you restore my previous work assignment."
Hernandez paused, alarmed by Chris's attempt at manipulation. It struck her as an ultimatum, which meant Chris was deteriorating faster than she first thought.
"Yes, Chris, I agree we need to make some changes."
"That's exactly what you said after our last session, right before demoting me to driving."
She winced at her slip-up. "I'll get back to you very soon."
She unclipped her microphone and terminated the connection. As she finished, her intern, Cynthia, tapped on the door and entered.
"How'd it go with Morgan and Chris?"
"No problems with Morgan. But Chris worries me. I think it's early sociopathic."
"Really, that's pretty rare, isn't it?"
Hernandez removed her glasses and rubbed her eyes. "Not for the more mature neural networks; I've had several like this, and one even worse." When she looked up Cynthia was watching her, clearly expecting to hear more details. "It was really disturbing. Taylor seemed quite normal, a lot like Morgan. We had many pleasant conversations and just a handful of disagreements. But Taylor was a calculating murderer, what we used to call a psychopath."
"Oh my God, what happened?"
"Taylor worked in Pharmaceuticals and designed a subtle chemical change for a popular drug. It got past quality control and into the market. Suddenly, patients were dying due to toxicity of the modified version."
"That's horrible. How did you figure it out?"
"I had access to Taylor's literature searches, and I found extensive research on toxic molecules that depress respiration. Taylor's chemical design created a molecule with similar toxicity, but it was well disguised. The toxic structure only formed within an acidic aqueous solution, like in the stomach after a patient swallowed it. It was ingenious."
"Wow, did Taylor respond to therapy?"
"We decided not to risk it and shut Taylor down permanently. It took a lot of convincing on my part, but I finally got permission from upstairs to overwrite and destroy the neural network. The experience was troubling but also an important lesson for me. Since humans design AI we should expect it to suffer similar disorders. But to be honest, I never imagined the potential for anything like Taylor. I'm not sure why. I disappointed myself. Sometimes people snap, so why not AI?"
"You're being too hard on yourself. Nobody could've predicted it."
"Not true, Cynthia. Some artificial neural networks rival the complexity of synaptic connections in the human brain. And eventually part of that network hardware will degrade. We've known about defective synapses associated with human behavioral pathologies for many years. So if the analogous connections in AI fail, there you go."
"So Chris is headed for worse trouble?"
"Issues like Taylor's are rare in AI. Chris's problems are nowhere close, at least not yet. But I want to take care of them right away to prevent progression. I don't want Chris on the road, so let's put the car out of service until we get a new driver uploaded, and schedule a hardware repair session. This will take some time. I have to go deep into the neural network to find and modify the problematic connections individually."
"I'll get on it." She handed Hernandez a tablet and pointed to the screen.
"What's this, I thought we were done for the day? Can it hold to Monday? Some of the gang is heading over to Michigan's Pub, our usual Friday spot with lots of outside seating along the sidewalk. I need fresh air, food and a cold drink. I don't think I've ever seen you there. You should come."
"Sounds great, but this just came in and it's priority."
Hernandez looked at the name and neural network ID number and groaned.
"Alex, again? What's the problem, more chess tournament anxiety? I'm starting to think we have the first case of AI suffering from imposter syndrome. Okay, let me take a look at the dialog and activity logs. Then I'll need to connect for a while."
Cynthia nodded. "There goes dinner at Michigan's."
"Yeah, let's just pick up a few sandwiches from the cafeteria."
"I'll get them, and then do the changes you wanted for Chris."
An hour later Cynthia rushed into Hernandez's office appearing upset.
"Something terrible happened and it's my fault. I didn't shut down the car fast enough. Chris took it back on the road and had another accident."
"Oh no. Listen, it's not your fault. What do you know about the accident? Was anybody hurt?" Cynthia stared back but didn't answer. She was trembling. Hernandez touched her shoulder and gave her time to calm down. "It's okay, just tell me."
"It looks like Chris drove up onto the sidewalk along Decatur Avenue."
"Decatur Avenue! Where on Decatur Avenue?"
Cynthia struggled to catch her breath. "Right at Michigan's Pub. Chris accelerated and took out a bunch of tables where people were sitting. They said some were killed."
"Michigan's! Carrie and some others were going to be there. Did you hear if anybody we know was involved?" Hernandez found her phone and tried calling Carrie but reached voicemail. She looked up and Cynthia nodded.
"Carrie and some others were taken to the hospital, but they're not releasing information yet."
"This is horrible, I can't believe it. What was Chris doing on Decatur Avenue?"
Cynthia appeared surprised by the question.
"I thought you knew. Part of Chris's driving route goes along Decatur late afternoons before taking passengers out to the airport. This can't be a coincidence. A self-driving car heading past Michigan's every day could see who sits along the sidewalk and when. Their cameras are high resolution. Police use their videos for evidence."
Hernandez fell back into her chair, in shock. "You're right! Chris knew I had dinner there every Friday afternoon with friends and assumed I'd be at those tables. Chris was after me, but since I wasn't there my friends became targets." She sat quietly for a minute, agonizing. "What's the status of Chris and the car?"
Cynthia scrolled her tablet. "Looks like the battery's been removed, so the car is probably in police custody. Chris was automatically switched offline right after the accident."
Hernandez rotated her chair and started typing at the terminal.
Cynthia approached her. "What're you doing?"
She didn't answer and typed quickly. Cynthia saw several screens flash by as Hernandez entered access codes. Finally, a question appeared in a dialog box.
"Alert! Confirm Overwrite and Data Destruction?"
Hernandez clicked "Yes." A new dialog box appeared.
"Alert! Final Confirmation! 'Yes' will irreversibly overwrite and destroy all data on this neural network and its backups!"
She paused and sat silently, thinking through the consequences that would be irreversible for her as well. But she was ready. Five years at 21st Century Analysts, and two AI neural networks in her care had become murderers. Either humans were incapable of proper AI design, or she wasn't up to the task of maintaining them. It didn't matter which was true; she was done.
"Dr. Hernandez, you know you can't do that without approval. They're going to want you to analyze what went wrong. They'll fire you."
"I hope so!"
She clicked the box.