Google’s long-standing dominance in internet search is increasingly extending into artificial intelligence, as the company’s search infrastructure and web-crawling capabilities feed its growing portfolio of AI products, including Gemini. The shift is raising concerns among publishers, regulators, and technology companies over whether Google’s control of search is giving it an unfair advantage in the rapidly evolving AI market.
For years, Google’s web crawler has indexed websites to deliver relevant search results and, in return, direct users to publishers’ pages. However, the growing integration of search and AI has blurred the distinction between content collected for traditional search and information used to generate AI-powered answers. Google’s crawler is now also helping support its AI systems, changing the economic relationship between search platforms and content creators.
According to Cloudflare CEO Matthew Prince, Google’s crawler sees significantly more of the web than comparable crawlers operated by OpenAI and Microsoft. This extensive reach gives Google access to a vast pool of online information that can strengthen its AI capabilities, while publishers face challenges in preventing their content from being used for AI without potentially losing valuable search visibility.
The issue has become more significant as AI-generated answers increasingly summarize and reproduce information without necessarily sending users back to the original websites. Cloudflare estimates that AI agents accounted for more than 57% of web traffic in 2026, surpassing human-generated traffic for the first time. This shift could weaken the economic incentives for publishers and creators to produce original content if their work is primarily consumed by machines rather than human readers.
Regulators are beginning to respond. The UK’s Competition and Markets Authority (CMA) has directed Google to give website owners the ability to prevent their content from being used in AI products while continuing to appear in traditional search results. Google has confirmed that it is testing such a setting and plans to roll it out globally after the UK testing phase.
The development marks an important moment in the evolution of the internet, where the competitive advantage of controlling search increasingly overlaps with the ability to build powerful AI systems. With Google still accounting for around 90% of the global search market, the debate over how its search infrastructure can be used for AI is likely to become a central issue in technology regulation and the future economics of online content.
