Artificial intelligence is killing the internet: How can it finally be saved?

There’s a growing fear among top executives at major media companies that artificial intelligence (AI) is “killing” the internet, or at least altering it in significant ways.

In the early part of last year, Matthew Prince, whose company Cloudflare provides security infrastructure for roughly 20% of the world’s web, began receiving worried calls from executives at major media outlets. They were grappling with a new and serious threat in the digital landscape.

When Prince asked if the threat was cyber-related, they responded, “No, it’s artificial intelligence.”

These executives had noticed the beginnings of a trend that has now become evident: AI is reshaping how people use the internet. Users are now turning to chatbots rather than traditional search engines for answers, receiving direct responses instead of links. This shift has led to a disturbing drop in traffic for content publishers, including news outlets, online forums, and sites like Wikipedia.

As AI transforms the way we surf the web, it’s also altering the economics of the internet. Online advertising was traditionally tied to human traffic, but with the decrease in this traffic, content publishers are desperately seeking new ways to compel AI companies to pay for their information. If they fail, the open web could morph into something entirely different.

Artificial Intelligence: The Emergence of ChatGPT and Its Impact on Information Search

The launch of ChatGPT by OpenAI in late 2022 drastically changed the way we search for information. The chatbot, which is now the most popular app in the iPhone App Store, is used by about 800 million people worldwide. Apple reported that for the first time in April, traditional searches in its Safari browser fell as people began to prefer asking the AI directly. OpenAI is even expected to release its own browser soon.

As OpenAI and similar companies continue to grow, Google, which commands about 90% of the conventional search market in the US, has integrated AI elements into its search engine. Last year, it started displaying AI-generated summary answers in some searches, a feature now prevalent in almost all searches. In May, it introduced “AI mode,” a chatbot version of the engine, advertising that users can now “let Google do the ‘googling’ for them.”

However, as Google takes over the “googling”, fewer people are visiting the websites that provide the answers. According to Similarweb, a company that monitors traffic to over 100 million domains, global search traffic (from humans) dropped by about 15% in the year leading up to June. While some sectors, like hobby blogs, are holding steady, others are taking a significant hit.

The categories suffering most from this shift are those traditionally reliant on search queries: science and education websites have seen a 10% drop in visitors, while health-related sites experienced a staggering 31% decline.

This has serious implications for companies that earn their revenue through advertising or subscriptions. “We’ve enjoyed a positive relationship with Google for years. But they’ve changed the game,” says Neil Vogel, CEO of Dotdash Meredith, which owns popular titles like People and Food & Wine. Three years ago, over 60% of traffic to the group’s sites came from Google; now it’s dwindled to 30%. “They’re using our content to compete with us,” Vogel asserts. Google, on the other hand, maintains that their use of content is fair. However, since the introduction of AI summaries, the proportion of news searches that don’t result in a site visit has surged from 56% to 69%, according to Similarweb.

In light of the threat of losing traffic and revenue, many major content creators have resorted to negotiating licensing deals with AI companies, and even engaging in legal disputes – a tactic cheekily referred to as “wooing and suing”. News Corp (owner of The Wall Street Journal and New York Post) has entered into a deal with OpenAI, while two of its subsidiaries have taken legal action against Perplexity, another AI answering service. The New York Times also brokered a deal with Amazon while simultaneously suing OpenAI.

Artificial Intelligence: Deals and Disputes

.

Thus, content publishers are attempting to secure fees from AI companies through agreements or litigation – but current laws often side with AI companies when it comes to using content for model training (fair use). A few efforts, such as Reddit’s deal with Google, have resulted in a significant annual sum, but this has not offset the loss in traffic or sales.

The majority of sites are too small to negotiate fees or engage in lawsuits. Initiatives like Cloudflare’s “pay as you crawl” and Tollbit’s paywall for bots aim to impose access fees on AI bots. Another model seeks to distribute advertising revenue from responses based on information gathered from third parties.

Meanwhile, content creators are reevaluating their own models. “The future of the internet is not solely about traffic,” suggests Chandrasekar of Stack Overflow, which has ramped up the platform’s enterprise subscription plans. News organizations are devising “Google zero” strategies (zero dependence on Google), promoting newsletters, apps, and shifting content behind subscriptions or live events. Despite a drop in referrals from Google, Dotdash Meredith has seen an overall rise in traffic. Audio and video content, which are legally and technically more challenging for AI engines to utilize compared to text, are also gaining traction. It’s no surprise that the site most frequently referred to by search bots is YouTube.

Despite these challenges, not everyone believes the internet is in decline – quite the opposite. Google’s Robby Stein contends that the internet is currently in “an extremely expansive phase”. Artificial Intelligence (AI) has simplified content creation, leading to a 45% surge in new websites over the past two years. AI-powered search queries have introduced innovative features, such as scanning images of your bookshelf to suggest what to read next, potentially driving more traffic. AI bots have also expanded the scope of “reading” websites – not by humans, but by machines. These bots can scan hundreds of pages to formulate answers, drawing from a wider range of sources than the average user.

Artificial Intelligence and the Decline in Web Traffic: Possible Explanations

While some might point fingers at Google for the drop in web traffic, Stein argues that they haven’t seen a significant decline in outbound clicks. The downturn could be attributed to other factors, such as users spending more time on social media or tuning into podcasts.

The concept of the “internet obituary” has been floated before, often citing the rise of social media or apps, but without concrete evidence. Yet, it seems that artificial intelligence may pose the most substantial threat to date. If the internet is to retain its current form, websites will need to explore new revenue models. “There’s no question that people prefer AI searches,” Gross states. “And for the internet, democracy, and content creators to thrive, AI must share its revenue with those who generate content.”

Information sourced from The Economist