Cloudflare and GoDaddy have come together to address the impact of AI on web traffic and revenue. They have introduced an AI Crawl Control tool to manage interactions with AI bots on websites, aiming to establish a system that requires permission for AI-driven web content. This collaboration seeks to combat the disruption faced by the internet’s traditional business model due to the rise of AI.

GoDaddy will incorporate Cloudflare’s AI Crawl Control into its web hosting platform, providing website owners with the ability to regulate how AI bots engage with their content. Site owners can choose to grant access to AI bot crawlers, block them, or potentially charge for access. These bots, operated by major AI companies, scour the internet for content without compensating creators, which has led to a decline in revenue for many publishers and small businesses.

Cloudflare manages traffic for approximately 20% of the web, while GoDaddy is the largest domain name registrar globally. Together, they possess the capability to influence online business practices. Additionally, both companies support new protocols like Agent Name Service and Web Bot Auth to authenticate AI agents and enhance transparency in their actions.

The ultimate objective is to establish a system where creators have better control over their content and can participate in the value generated by AI systems. By creating a permission-based framework for the AI-driven web, Cloudflare and GoDaddy aim to prevent bots from freely scraping content and ensure that creators benefit from their work. Failure to address this issue could lead to a decrease in original content creation as AI systems continue to extract value without directing traffic back to the source.

Cloudflare and GoDaddy are optimistic that by focusing on identity verification, control mechanisms, and fair compensation, they can lay the foundation for a sustainable economic model in the evolving landscape of the internet.