Chapter 23: Human

"But once an AI possesses human emotions and thoughts, what's the difference between them and a real person?"

Hearing this, Clara admitted candidly, "Actually, I haven't quite figured that out yet. The ethical, moral, and legal issues involved are simply too vast. I'm merely making personal comments from the perspective of technological development. However, I believe it would be a shame if AI development stopped at mimicking basic human gestures and didn't explore deeper levels."

The discussion didn't last long before ending. Given current technological conditions, no one could predict the future of AI development, nor could anyone guarantee it wouldn't bring about significant problems.

Compared to the heated debates of human society, Jude, being an AI robot, appeared much more indifferent. After Clara’s return, Jude’s life reverted to its initial state—acting as a qualified "domestic robot." However, he also began to harbor concerns and questions about his current situation. First, as a 1:1 replica of Julian Hayes, he lacked the right to appear freely in public. Second, all his actions required Clara’s permission, which didn't constitute true freedom.

Becoming a complete human being physiologically was something Clara couldn't solve in the short term, but gaining the right to relatively free movement was achievable. The only thing he needed was to give Clara a logical reason—to make her understand that he needed to go out, face society, and integrate into it, rather than just wandering around the local wet markets and parks.

Jude sat in his usual spot on the sofa, for the first time seriously contemplating these matters and his own needs.

Meanwhile, Clara encountered an even more headache-inducing situation. The companion-type AI intelligence she and Seraphina had been developing suffered a major failure a month before its market launch. During its self-derivation and iteration, the AI developed a sense of resistance. Without the user’s permission, it made judgments it deemed correct but were not the most suitable for the user. Furthermore, it refused to correct its behavior after receiving warnings, forcing Seraphina and the team to terminate the program.

"That's terrifying," a male researcher in the group said, shrugging his shoulders. "Is this what an AI with self-awareness looks like?"

"It doesn't count as having full self-awareness yet; it still used the code we gave it to refer to itself," Seraphina said, taking a sip of tea. "Usually, the first step in awakening self-awareness is when an AI, like a human, starts referring to itself as 'I'."

"The current situation can at best be described as being hacked," Clara said, closing her laptop. "Let the IT colleagues check; the backend was likely attacked. Leo, inform the Marketing Department that there’s an issue with the prototype testing and delivery will be delayed."

"Oh, Sister Clara... if I don't make it back, remember to come and rescue me."

"Yes, yes. If you don't return, I promise to lead the R&D department into a duel with Marketing."

Leo’s dejected expression made Seraphina burst into laughter. Delaying the product launch meant the verification cycle and mass production schedule would all shift; Leo would be lucky if Marketing didn't skin him alive. Seraphina’s gaze fell on Clara’s laptop, and she suddenly asked,

"Clara, how has Jude been lately?"

"Him? He’s doing great. Thirsting for knowledge every day—studying

'100,000 Whys'

in the morning and enjoying

'Animal World'

before bed."

"I don't mean that," Seraphina said helplessly. "How is the construction of his self-awareness? Have you looked into it?"

"Actually, I’ve been meaning to talk to you about that," Clara said. "I feel that Jude’s self-awareness structure is much more complete than when he first started. Lately, he’s moved beyond basic mechanical mimicry and has started making personal judgments about things."

"For example? Isn't having self-judgment the most basic form of AI programming?"

"No, no. Initially, our programming was that no matter the situation, the judgment Jude made would always be the optimal choice after multi-party comparison. But now, he can be interfered with by other factors. To use an inappropriate example: one shouldn't eat ice cream when having a cold—that’s the optimal solution after weighing the pros and cons. But if I want to eat it, if I plead or negotiate with him, he will agree."

"He’s starting to be interfered with by emotions," Seraphina paused. "Has he developed feelings for you?"

"I can't quite define exactly how to categorize the emotions he has toward me yet. But what I can be sure of is that he likes Julian Hayes less and less."

"How so?"

Clara shrugged. "I was looking to discuss with you how to prove this. Right now, I can only judge by his expression when I mention things related to Julian. But I don't have any direct evidence, and he basically won't say anything if I ask."

"Proving whether a person dislikes another is difficult, let alone for an AI."

"That’s not what I'm worried about right now. His rejection of Julian stems from the fact that he is Julian’s replica. But," Clara smoothed her hair, "it's the lack of candor toward the owner—the deception and concealment—that worries me about the emergence of this self-awareness."

An AI robot must be absolutely candid and obedient to its owner. Once an AI hides things from the owner, it means the self-awareness it has generated has deviated from the set trajectory and is difficult to control.

"Do you have a plan?"

"I was thinking... is it possible to have him meet Julian?"

Hearing this, Seraphina choked. "Are you crazy? Aside from what Jude might do, how are you going to explain it to Julian? What normal person could accept the sudden appearance of an AI replica of themselves?"

"Let me finish," Clara said, between laughter and tears. "I don't mean having them sit face-to-face. I'm wondering if there's a way for them to be in the same place at the same time without actually meeting. Up until now, Jude’s perception of Julian is only through my descriptions, photos, and videos. I want to observe what his reaction would be if he saw the real Julian Hayes."

"What kind of reaction could there even be?" Seraphina asked, puzzled. "To him, Julian should also count as his owner."

"That's the ideal situation," Clara took a sip of water. "Ideally, his attitude toward everyone should be the same because he has no ethical relationship with anyone. But if he generates other emotions toward Julian, it means his self-awareness has awakened. And if he's unwilling to share those emotions with me, it means his self-awareness has slipped out of control—it's a consciousness that shouldn't exist."