The biggest names in AI are backing a new way to opt out of scraping

<figure> <img alt="Photo illustration of a brain on a circuit board in red." data-caption="" data-portal-copyright="Illustration by Cath Virginia / The Verge | Photos from Getty Images" data-has-syndication-rights="1" src="https://platform.theverge.com/wp-content/uploads/sites/2/chorus/uploads/chorus_asset/file/25330660/STK414_AI_CHATBOT_H.jpg?quality=90&#038;strip=all&#038;crop=0,0,100,100" /> <figcaption> </figcaption> </figure> <p class="wp-block-paragraph">OpenAI, Google, Microsoft, and Anthropic say they will adopt a new standard that publishers can use to opt out of being scraped by AI, as <a href="https://www.seroundtable.com/google-ai-training-control-42430.html">reported earlier by <em>Search Engine Roundtable</em></a>. With this control, publishers can apply their AI training preferences across multiple AI companies, instead of setting them individually.</p> <p class="wp-block-paragraph">The <a href="https://datatracker.ietf.org/doc/draft-kelley-aipref-training-preference-signal/">shared "policy token" signal</a> is proposed by Kevin Kelley, a technical lead for standards at Anthropic, as part of the Internet Engineering Task Force (IETF). It will live within a website's <a href="https://www.theverge.com/24067997/robots-txt-ai-text-file-web-crawlers-spiders">robots.txt file</a>, which has long signaled which web crawlers can and can't scrape a site - whether for search indexing, archiving, or …</p> <p><a href="https://www.theverge.com/tech/1023763/openai-google-microsoft-ai-scraping-standard-opt-out">Read the full story at The Verge.</a></p>