The TikTok algorithm ByteDance crown jewel: why Microsoft, Oracle and Walmart would suffer to replicate their magic - Experts assure Business Insider that the potential buyer of TikTok will suffer to replicate the magic of the app if it acquires the company's business in the United States without its recommendation algorithms.
ByteDance is in negotiations with several companies interested in the TikTok business in the United States after Donald Trump ordered the sale of the platform in August or, otherwise, would be exposed to a total veto. The companies leading the bid, which is currently between 20,000 and 30,000 million dollars (up to 25,000 million euros at the current exchange rate), range from Microsoft and Walmart to Oracle.
However, negotiations have stalled by new Chinese regulations that restrict the export of key services, including algorithms. The result is that an American firm could end up buying the app business in that country but without having access to its algorithms, which are precisely the ones that make the app so addictive. One of the great secrets of his success.
"I personally believe that TikTok would not be TikTok without its algorithm," explains Bondy Valdovinos Kaye, a researcher at the Queensland University of Technology (Brisbane, Australia), who has researched how TikTok works.
- Employees say the TikTok algorithm is the crown jewel
TikTok has captivated about 100 million users in the United States (in Spain the app adds more than 14 million downloads), and in appearance is not much different from YouTube. People upload short videos doing anything, whether they're taking part in viral challenges or posting comic sketches and memes.
TikTok algorithm ByteDance crown jewel
What sets the app apart is its "For You/FYP" page, which concentrates a seductive mix of highly viral and shareable random short videos. Like YouTube and Netflix-style apps, everything revolves around a system that recommends the user what to watch next. What's strange about TikTok is how good the app is at it.
- Valdovinos Kaye visited Bytedance offices in Beijing in 2019, where employees called the algorithm the" crown jewel " of ByteDance's success.
The algorithm was developed as a collaboration between ByteDance's Artificial Intelligence Laboratory and Peking University, according to Valdovinos Kaye, and is the secret recipe that feeds all of Bytedance's software.
Initially developed for Toutiao, the Bytedance news aggregator, it is now used in all versions of Bytedance applications, including Douyin, the Chinese version of TikTok.
"We haven't seen anyone else master the recommendations as successfully as they have," says Sabba Keynejad of Veed, a video editing app that has attempted to reverse engineer the algorithm.
Of course, ByteDance employs an immense engineering workforce to develop the algorithm.
When Valdovinos Kaye visited the offices in Beijing, he found a multi-storey office full of programmers. (TikTok was contacted to participate in this report but declined an interview request. However, in June they published an article on their blog explaining in detail how their algorithm works).
Keynejad suggests that we may be giving the algorithm more importance than it has.
"The algorithm is not what makes everything work, but I think it's like a perfect storm," he explains. "Your product arrives at the right time on the market, meets these unemployed teenagers with this great recommendation engine, and all these topics going through the app."
That's something Eugene Wei, a former product manager at companies like Flipboard and Amazon, agrees with. Wei has delved into the magic of TikTok algorithms in a well-known blog.
"It has really easy-to-use video filters and editing tools, combined with an algorithm that quickly puts them within reach of many people and generates feedback," he says. "That wheel is completely attached on all sides."
And he adds, " a lot of people are treating the algorithm for themselves as some kind of black magic."
"I don't think it's really that. Most people who build recommendation engines using machine learning say that the techniques they use are very likely to be quite standard."
Nikita Aggarwal, who works at the Oxford Internet Institute, adds that the algorithm learns from the huge amount of data that TikTok is able to use.
"It potentially collects more user data from other applications, and is therefore able to better profile the user and therefore recommend which videos are more likely for him to enjoy," he notes. She also agrees that the design of the app - and its focus on immersive full-screen videos-helps boost her popularity.
"Each click reveals the user's preferences, to the point that it gives TikTok more useful information about the user's preferences and lets you understand that that's what makes an application great," he says.
The application can also verify the interests of the user on a scale unimaginable for other platforms such as YouTube.
The short format of TikTok videos-no more than a minute-means that users can watch them at a much faster rate than on YouTube, where a conventional video can last more than 12 minutes.
"For YouTube it's harder because they don't necessarily have an app that puts a lot of random videos in front of a lot of people," Wei says. "People choose for themselves what they see on YouTube, which is fine..."
The TikTok it page offers its users hundreds of videos per hour, which means you can occasionally try videos that your users may not like with minimal impact.
"Feedback only takes a few seconds for me to record how I feel about a video, and since it's full-screen, the app can assume that anything it does reflects my opinion about that video," Wei says.
The real-life data that is used to refine the TikTok machine learning algorithm is, in turn, more powerful than the training datasets that use other services.
Even if it is a relatively standard algorithm, TikTok has been well trained with its nearly 600 million monthly active users, which could prove problematic if the algorithm in question is not sold.
"I think it's still possible to replicate much of the magic of the algorithm if you're going to give it all the user data and all the video data, but you'd have to be willing to support that whole process," Wei says.
"One of the things that comes up in this context is how you relate the data in the algorithmic system or the algorithm to the data it's trained on," Aggarwal adds.
"If it were sold to Microsoft, Microsoft would presumably benefit from the same data, but there is the knowledge that ByteDance has already acquired from users to whom it has already exposed the machine learning algorithm, and that is valuable.
"That training record has a legacy value that China clearly recognizes."
TikTok algorithm ByteDance crown jewel