Reddit has long been a hot spot for conversation on the internet. About 57 million people visit the site every day to chat about topics as varied as makeup, video games and pointers for power washing driveways.
In recent years, Reddits array of chats also have been a free teaching aid for companies like Google, OpenAI and Microsoft. Those companies are using Reddits conversations in the development of giant artificial intelligence systems that many in Silicon Valley think are on their way to becoming the tech industrys next big thing.
Now Reddit wants to be paid for it. The company said on Tuesday that it planned to begin charging companies for access to its application programming interface, or A.P.I., the method through which outside entities can download and process the social networks vast selection of person-to-person conversations.
The Reddit corpus of data is really valuable, Steve Huffman, founder and chief executive of Reddit, said in an interview. But we dont need to give all of that value to some of the largest companies in the world for free.
The move is one of the first significant examples of a social networks charging for access to the conversations it hosts for the purpose of developing A.I. systems like ChatGPT, OpenAIs popular program. Those new A.I. systems could one day lead to big businesses, but they arent likely to help companies like Reddit very much. In fact, they could be used to create competitors automated duplicates to Reddits conversations.
Reddit is also acting as it prepares for a possible initial public offering on Wall Street this year. The company, which was founded in 2005, makes most of its money through advertising and e-commerce transactions on its platform. Reddit said it was still ironing out the details of what it would charge for A.P.I. access and would announce prices in the coming weeks.
Reddits conversation forumshave become valuable commodities as large languagemodels, or L.L.M.s, have become an essential part of creating new A.I. technology.
A brave new world. A new crop of chatbotspowered by artificial intelligence has ignited a scramble to determine whether the technology could upend the economics of the internet, turning todays powerhouses into has-beens and creating the industrys next giants. Here are the bots to know:
ChatGPT. ChatGPT, the artificial intelligence language model from a research lab, OpenAI, has been making headlines since November for its ability to respond to complex questions, write poetry, generate code, plan vacationsand translate languages. GPT-4, the latest version introduced in mid-March, can even respond to images(and ace the Uniform Bar Exam).
Bing. Two months after ChatGPTs debut, Microsoft, OpenAIs primary investor and partner, added a similar chatbot, capable of having open-ended text conversations on virtually any topic, to its Bing internet search engine. But it was the bots occasionally inaccurate, misleading and weird responsesthat drew much of the attention after its release.
Ernie. The search giant Baidu unveiled Chinas first major rival to ChatGPT in March. The debut of Ernie, short for Enhanced Representation through Knowledge Integration, turned out to be a flopafter a promised live demonstration of the bot was revealed to have been recorded.
L.L.M.s are essentially sophisticated algorithms developed by companies like Google and OpenAI, which is a close partner of Microsoft. To the algorithms, the Reddit conversations are data, and they are among the vast pool of material being fed into the L.L.M.s. to develop them.
The underlying algorithm that helped to build Bard, Googles conversational A.I. service, is partly trained on Reddit data. OpenAIs Chat GPT cites Reddit data as one of the sources of information it has been trained on.
Other companies are also beginning to see value in the conversations and images they host. Shutterstock, the image hosting service, also sold image data to OpenAI to help create DALL-E, the A.I. program that creates vivid graphical imagery with only a text-based prompt required.
Last month, Elon Musk, the owner of Twitter, said he was cracking down on the use of Twitters A.P.I., which thousands of companies and independent developers use to track the millions of conversations across the network. Though he did not cite L.L.M.s as a reason for the change, the new fees could go well into the tens or even hundreds of thousands of dollars.
To keep improving their models, artificial intelligence makers need two significant things: an enormous amount of computing power and an enormous amount of data. Some of the biggest A.I. developers have plenty of computing power but still look outside their own networks for the data needed to improve their algorithms. That has included sources like Wikipedia, millions of digitized books, academic articles and Reddit.
Representatives from Google, Open AI and Microsoft did not immediately respond to a request for comment.
Reddit has long had a symbiotic relationship with the search engines of companies like Google and Microsoft. The search engines crawl Reddits web pages in order to index information and make it available for search results. That crawling, or scraping, isnt always welcome by every site on the internet. But Reddit has benefited by appearing higher in search results.
The dynamic is different with L.L.M.s they gobble as much data as they can to create new A.I. systems like the chatbots.
Reddit believes its data is particularly valuable because it is continuously updated. That newness and relevance, Mr. Huffman said, is what large language modeling algorithms need to produce the best results.
More than any other place on the internet, Reddit is a home for authentic conversation, Mr. Huffman said. Theres a lot of stuff on the site that youd only ever say in therapy, or A.A., or never at all.
Mr. Huffman said Reddits A.P.I. would still be free to developers who wanted to build applications that helped people use Reddit. They could use the tools to build a bot that automatically tracks whether users comments adhere to rules for posting, for instance. Researchers who want to study Reddit data for academic or noncommercial purposes will continue to have free access to it.
Reddit also hopes to incorporate more so-called machine learning into how the site itself operates. It could be used, for instance, to identify the use of A.I.-generated text on Reddit, and add a label that notifies users that the comment came from a bot.
The company also promised to improve software tools that can be used by moderators the users who volunteer their time to keep the sites forums operating smoothly and improve conversations between users. And third-party bots that help moderators monitor the forums will continue to be supported.
But for the A.I. makers, its time to pay up.
Crawling Reddit, generating value and not returning any of that value to our users is something we have a problem with, Mr. Huffman said. Its a good time for us to tighten things up.
We think thats fair, he added.
Read more:
Reddit Wants to Get Paid for Helping to Teach Big A.I. Systems - The New York Times
Recommendation and review posted by G. Smith
