The future has arrived for livestreaming

A current MIT Technology Review article reports on the new and rather scary developments in AI generated clones.

For those who have recently scrolled through the livestreaming videos at 4 a.m. Beijing time on Taobao, might have wondered about the huge number of still very diligent streamers presenting products and offering discounts. Looking more closely, one can notice that many of these livestream influencers seem slightly robotic. The movement of their lips largely matches what they are saying, but there are always moments when it looks unnatural.

That’s because these streamers are not real: they are AI-generated clones of the real streamers.

Today, livestreaming is the dominant marketing channel for traditional and digital brands in China. However, training livestream hosts, retaining them, and figuring out the technical details of broadcasting comes with a significant cost for smaller brands. It’s much cheaper to automate the job.

Since 2022, Chinese startups as well as major tech companies have been offering the service of creating deepfake avatars for e-commerce livestreaming. By now it needs not more than one minute of sample video and about 1,000 USD to clone a human streamer to work 24/7.

The Nanjing-based startup Silicon Intelligence is offering a digital clone that could speak and act like a human for these conditions.

Once the avatar is generated, its mouth and body move in time with the scripted audio. Even the script is done by AI. All the human workers have to do is input basic information such as the name and price of the product being sold, proofread the generated script, and watch the digital influencer go live.

A more advanced version of the technology can spot live comments and find matching answers in its database to answer in real time, so it looks as if the AI streamer is actively communicating with the audience.

Xiaoice, another company, has always been focused on creating more human-like AI, particularly avatars that are capable of showing emotions. They also offer their services for less than one thousand USD. One of their customers was Liu Jianhong, a Chinese sports announcer, who had a clone of himself to read out the match results and other relevant news on Douyin during the 2022 FIFA World Cup.

Actually, Douyin banned the use of recorded videos as livestreams. AI-generated livestreaming, with no recorded footage but also little real-time human input, straddles the line on that rule.

To use such technology might also be useful for marketing for outbound tourism. However, it is quite scary to think about what abuse is possible with this technology which gives everybody the chance to create fake videos of convincing-looking avatars speaking whatever the scripts says.