Since July 1, Reddit posts have stopped appearing on Bing and other search engines besides Google. After an investigation, Microsoft confirmed that this is due to Reddit blocking these platforms.
The likely reason for this blockage is a recent agreement with Google. As part of this agreement, Reddit allows Google to use its platform data to train Google’s artificial intelligence algorithms.
Reddit blocks indexing of content to Bing and other search engines other than Google
For the past few weeks, users trying to search for recent content on Reddit will find that search engines like Bing don’t index this content. For example, if you search for all content on Reddit from the last week, it will appear completely empty on Bing. This is because Reddit updated its robots.txt file, which tells bots what content or directories should not be indexed. This file helps protect certain parts of a website from being easily found by search engines.
Search Engine Land reached out to Microsoft for more information, and a company spokesperson confirmed the block with the following statement:
“We respect the robots.txt rule. Bing stopped crawling Reddit after they implemented their updated robots.txt file on July 1st, which prohibits all crawling of their site.”
Google deal could be behind the blockade
This blockage comes months after an agreement between Reddit and Google. In this agreement, Google is allowed to use data stored on Reddit to train its artificial intelligence algorithms, in exchange for a payment of $60 million to the platform.
The Verge reached out to Reddit for an explanation regarding the reasons for the block. A spokesperson for Reddit denied that the block is directly related to the agreement to use Reddit data for training Google’s algorithms.
“This is completely unrelated to our recent partnership with Google. We have been in discussions with multiple search engines. We have not been able to reach agreements with all of them, as some are unable or unwilling to make enforceable promises regarding their use of Reddit content, including its use for AI.”
In the statement, Reddit implied that it has attempted to build partnerships with various companies regarding the treatment of its content. They reached an agreement with Google earlier this year. However, they did not reach a commercial agreement with Microsoft, leading Reddit to block the indexing of its content on Bing.
For users of Microsoft’s search engine, the only solution for now is to use Google when searching for recent information on Reddit. This block by Reddit does not seem likely to be resolved anytime soon.
