What happens when the robots sound too much like humans?

Artificial intelligence has a new challenge: Whether and how to alert people who may not know they’re talking to a robot.

At Google I/O 2018, the company showed off a computer assistant that makes convincingly human-sounding phone calls, at least in its pre-recorded demonstration. But the real people in those calls didn’t seem to be aware they were talking to a machine. That could present thorny issues for the future use of AI.

Among them: Is it fair — or even legal — to trick people into talking to an AI system that effectively records all of its conversations? And while Google’s demonstration highlighted the benign uses of conversational robots, what happens when spammers and scammers get hold of them?

Google CEO Sundar Pichai elicited cheers as he demonstrated the new technology, called Duplex, during the company’s annual conference for software developers. The assistant added pauses, “ums” and “mmm-hmms” to its speech in order to sound more human as it spoke with real employees at a hair salon and a restaurant.

“That’s very impressive, but it can clearly lead to more sinister uses of this type of technology,” said Matthew Fenech, who researches the policy implications of AI for the London-based organisation Future Advocacy. “The ability to pick up on nuance, the human uses of additional small phrases — these sorts of cues are very human, and clearly the person on the other end didn’t know.”

Fenech said it’s not hard to imagine nefarious uses of similar chatbots, such as spamming businesses, scamming seniors or making malicious calls using the voices of political or personal enemies.

“You can have potentially very destabilising situations where people are reported as saying something they never said,” he said.

Pichai and other Google executives tried to emphasise that the technology is still experimental, and will be rolled out cautiously. It’s not yet available on consumer devices.

“It’s important to us that users and businesses have a good experience with this service, and transparency is a key part of that,” Google engineers Yaniv Leviathan and Yossi Matias, who helped design the new technology, wrote in a blog post. “We want to be clear about the intent of the call so businesses understand the context. We’ll be experimenting with the right approach over the coming months.”

It’s unclear how the company will navigate existing telecommunications laws, which can vary by state or country. Google didn’t return a request for comment on how it plans to seek the consent of people called by its bots.

One co-owner of a San Francisco Bay Area barbershop patronised by some Google employees was a little creeped out by the privacy implications.

“It seems like something that would be helpful for our clients,” said Katherine Esperanza, co-owner of the Slick and Dapper barbershop in Oakland, California. Esperanza, however, wondered if the shop would be able to block the calls, and said it “begs the question about whether the conversation is recorded and if the recipient of these automated calls could be aware that they’re being recorded.”

Anti-wiretapping laws in California and several other states already make it illegal to record phone calls without the consent of both the caller and the person being called. The Federal Communications Commission has also been grappling with rules for robocalls, the unsolicited and automatically-dialed calls made by telemarketers.

Such calls are typically prerecorded monologues, but more businesses and organisations are employing machine-learning techniques to respond to a person’s questions with a natural-sounding conversation, in hopes they’ll be less likely to hang up.