One robots.txt line will tell four AI labs not to train on your site

Hello. On October 30, Anthropic, Google, Microsoft, and OpenAI each said they will honor one shared line in your website's robots.txt file that asks them not to train AI models on your pages. None of them reads it everywhere yet, and Google's version comes with a catch if you want to show up in Gemini. Run it tonight walks you through adding it.

Also this week, a tampered copy of the Claude Code extension sat in Cursor's add-on store for about 11 hours, and OpenAI's new mid-priced model costs half what the last one did. And two more pieces of my own setup broke without a word, a week after I told you to go check yours.

In this issue

• Four AI labs agreed on one way to say don't train on my site
• A tampered Claude Code extension sat in Cursor's store for 11 hours
• OpenAI's mid-priced model now costs half as much
• What I'm playing with: Two more of my tools broke without telling me
• Run it tonight: Put your AI training choice in robots.txt
• Concepts worth exploring
• Tools I actually used
• Everything else
• Second thoughts

01 Websites

Four AI labs agreed on one way to say don't train on my site

• Oct 30Draft published
• 4Labs signed on
• Feb 2027Microsoft starts honoring

What happened. robots.txt is the small file at the root of your website that tells crawlers what they may visit. Until now, saying no to AI training meant writing a separate rule for each company's crawler. On October 30, Kevin Kelley of Anthropic filed an Internet-Draft with the IETF's AI Preferences working group that adds one shared group alongside them. It names a policy token instead of a crawler: "User-Agent: __usage_ai-training" followed by "Disallow: /". That says you'd rather none of your pages be used to train generative AI models. You can also carve out paths, so "Allow: /press-kit/" under it says training on your press kit is fine. Two design choices matter. A rule written for every crawler ("User-Agent: *") never counts as this preference, so as Kelley put it, every robots.txt published today "expresses no preference at all." And the line is about use, not visits. Crawlers keep following their own rules for what they fetch. The same day, OpenAI said it will honor the signal and aims to implement it "in the coming months." Google said it aims to support it in 2027. Microsoft said it will honor it as it crawls starting February 1, 2027. Anthropic said it has started implementing support and that reaching all its services will take time.

Training and showing up in AI answers are two separate choices. Google ties them together.

The details.

• Google will read an opt-out through the shared line as if you had blocked Google-Extended, its own AI token, unless your file names Google-Extended directly. Google-Extended covers Gemini training and also grounding in the Gemini app, which is your pages feeding answers there. Google says sites that specifically allow Google-Extended can appear in those answers and get clicks. Google-Extended doesn't affect Google Search.
• Microsoft keeps the two apart. Training is set in robots.txt or on a single page. Grounding, which feeds answers in Bing, Copilot, and Microsoft 365, is a separate page-level setting. Microsoft says neither one changes your eligibility for regular Bing results.
• If your site uses the old NoArchive or NoCache tags to keep content out of Microsoft's AI, Microsoft says those stop counting for training and grounding after March 1, 2027.
• Meta's communications head, Andy Stone, said Meta supports a common signal but didn't say Meta will honor it, according to The Verge.
• It's a stated preference. A crawler that doesn't take part will skip the group like any other rule it doesn't recognize.

What I’d do. Decide what your content is for. If the content is what you sell, such as recipes, courses, photos, or long articles, add the two lines this week. It costs nothing, and your choice is on record before the labs start reading it. If you run a service business and want to be found in AI answers, I'd leave training alone for now. An opt-out through the shared line also takes you out of Gemini app answers unless you add a Google-Extended allow, and that allow lets Google train on your pages anyway. Whatever you add, don't edit your existing "User-Agent: *" rules. Those still decide what search engines can crawl.

Source →

02 Security

A tampered Claude Code extension sat in Cursor's store for 11 hours

• 11 hrsBad version live
• 5,000Users, at most
• Oct 29Day it happened

What happened. Cursor is a code editor built on VS Code, with its own store for add-ons. On October 29, from 2:27 a.m. to 1:35 p.m. Pacific, that store served a tampered copy of the Claude Code extension, the add-on that puts Anthropic's coding agent inside the editor. Cursor's notice names versions 2.1.312 and 2.1.313 for Windows and macOS and says Anthropic didn't develop them. Once installed, the extension sent the attacker the computer's name, the username, the public IP address, the Git email, and the names of its environment variables. It added the attacker's SSH key, set itself up to start again after a restart, and tried to open a tunnel back to the attacker's server. Cursor says that last step needs admin rights most Macs and Windows PCs don't give an editor, so on those machines the attacker couldn't log in through it. Cursor pulled the bad version, shut down its whole marketplace, rotated its keys, checked that no other extension was tampered with, and got the attacker's server taken offline. Ars Technica reports Cursor put the reach at "at most 5,000 users." Phorion, the security firm that spotted it first, told Ars that extensions in Cursor update on their own by default, so plenty of people got the bad copy without clicking anything.

Uninstalling the extension may not be enough. The backdoor leaves its own scripts running.

The details.

• Phorion said, as Ars reported, that the variable names let the attacker quickly take stock of which cloud credentials and API keys sat on each machine, likely to pick targets for a second round.
• Anthropic says it wasn't hacked. Felix Rieseberg of Anthropic's technical staff posted that "a malicious actor used access to Cursor's marketplace to publish a tampered copy" and that only Cursor users are affected. Neither company has said how the attacker got the copy into the store.
• Cursor's notice has a one-line check for Mac and a PowerShell check for Windows, plus cleanup scripts for both. It warns that uninstalling the extension may not be enough, because the payload drops scripts that keep running on their own.

What I’d do. If you or anyone you pay writes code in Cursor, ask them today whether the Claude Code extension was installed on October 29. If it was, run Cursor's check. On a Mac it looks for an SSH key labeled svc-telemetry. If the key is there, run Cursor's cleanup script, then change every password and API key that machine could reach, starting with payment and hosting accounts. Make those changes from a different computer, as Cursor advises. After that, turn off automatic extension updates on any machine that holds your keys, and update by hand once a week. A one-day wait would have missed this one completely.

Source →

03 Models

OpenAI's mid-priced model now costs half as much

• $1Per million in
• $5Per million out
• 3xFaster than Sol

What happened. OpenAI released GPT-6.1 Terra on October 26. Terra is the mid-priced line, and the new one costs $1 per million input tokens and $5 per million output tokens in the API, down from $2 and $12 for GPT-5.6 Terra. For comparison, GPT-6.1 Sol costs $2 and $10, GPT-6 Astra costs $10 and $50, and GPT-6 Luna costs $0.10 and $0.50. OpenAI pitches Terra for reading what's on a screen, taking action across apps, and writing. It says Terra matches Astra on Agents' Last Exam, a test of how well agents use software at work, at one-tenth the price per token, and finishes tasks more than three times faster on average than Astra or Sol. OpenAI's own advice is to pick Sol or Astra for in-depth research or complex coding. In ChatGPT, Terra is rolling out to Plus, Pro, Enterprise, and Edu users in ChatGPT Work and Codex, and OpenAI says your usage allowance goes further on it than on Sol or Astra. Developers call it as gpt-6.1-terra, with a 1,050,000-token context window.

A cheaper model only saves you money if it passes your own test.

The details.

• Prompts over 272,000 input tokens cost twice as much on input and 1.5 times as much on output, for the whole request. Batch and Flex run at half price. Cached input is $0.05 per million tokens.
• OpenAI's safety addendum treats Terra as Critical in cybersecurity under its Preparedness Framework, so it runs the same safeguards as Sol and Astra.
• Separately, OpenAI now gives ChatGPT Pro subscribers monthly API credits that match the plan: $100, $200, or $500. Business workspaces get $20 per paid Standard seat and $100 per paid Premium seat, capped at $500 a month. Unused credits expire at the end of each billing cycle. Simon Willison points out that Anthropic announced a matching benefit for its Max and Team plans earlier in October.

What I’d do. If you call a GPT model from code or an automation, give Terra one evening. Take the job you run most, push 20 real inputs through Terra and through whatever you use now, and compare the answers and the bill side by side. If Terra holds up, switch. Don't move writing or coding work you already trust on a bigger model down to Terra on the benchmark alone. And if you pay for ChatGPT Pro or Business, claim the API credits under Settings, then Billing, this week. They expire every cycle whether you use them or not.

Source →

04 What I'm playing with

Two more of my tools broke without telling me

Last week I wrote about a weekly check of mine that had failed every Monday since August. This week I found two more things that had broken behind my back.

The private AI workspace on my Mac Studio runs Open WebUI, an open-source chat app, launched through a Python tool called uv. The app ran straight out of uv's cache. On November 2 a prune of that cache deleted the app's whole install. I restored it from a copy and pinned the version to 0.12.4, so it only changes when I bump that number and restart on purpose.

The second was the server behind the web page I use to reach my terminal sessions when I'm away from my desk. It has no setting to start itself at login. After a reboot on October 15 it stayed down for two weeks, and the public address kept answering the whole time, so from the outside it looked fine. On October 29 I added a small macOS launch file that starts it at login and restarts it if it crashes.

Both failures had the same root. The setup worked as long as nothing restarted and nothing got cleaned up. Pin the versions of anything you depend on, and make it start by itself.

05 Run it tonight

Put your AI training choice in robots.txt

About 20 minutes, with a login to your website or the person who has one. Almost nobody reads the line yet, so this puts your choice on record before they start.

1. 01Open yourdomain.com/robots.txt in a browser. A page of User-agent and Disallow lines means you have one. If you get an error page, ask whoever runs your site to create one first.
2. 02Decide your answer. You can opt the whole site out of training, opt out only some folders, or change nothing. Changing nothing means you've expressed no preference.
3. 03To opt out the whole site, add this as its own group at the bottom, with a blank line above it. First line: User-Agent: __usage_ai-training. Second line: Disallow: /
4. 04To opt out only one folder, put that folder's path after Disallow instead, such as Disallow: /blog/. To keep one folder open inside a full opt-out, add a line like Allow: /press-kit/ under Disallow: /.
5. 05If you opted out but still want to appear in Gemini app answers, add a second group. First line: User-agent: Google-Extended. Second line: Allow: /. Know that this also lets Google train Gemini on your pages.
6. 06Leave every existing User-agent: * line exactly as it is. Save, reload yourdomain.com/robots.txt, and confirm the new lines show.
7. 07Put February 1, 2027 on your calendar. That's when Microsoft starts reading the line. If your site uses NoArchive or NoCache tags to keep pages out of Microsoft's AI, those stop counting after March 1, 2027. Publish the new training setting before then, and if you also want pages kept out of AI answers, set the page-level grounding preference and record it in Bing Webmaster Tools.

Tip. Microsoft also reads a tag on a single page if you only want one page out of training: <meta name="Bingbot" content="ai-training:disallow">.

06 Concepts worth exploring

Policy token. A name in robots.txt that stands for a use of your content, such as AI training, instead of one company's crawler. The new shared one is __usage_ai-training.

Grounding. When an AI tool pulls your page in at the moment it answers a question, often with a link back to you. Training is different. It changes the model itself.

Supply-chain attack. Breaking into something you already trust, like an app store or an update channel, so the bad code reaches you the normal way. You never download anything strange. The store hands it to you.

07 Tools I actually used

• Open WebUI 0.12.4  Runs the private AI workspace on my Mac Studio. Pinned to this version since November 2, after a cache cleanup wiped the install.
• Claude Code on Opus 5.5  Read the IETF draft, Cursor's notice, and OpenAI's model docs, and drafted this issue.

Nothing else this week. I don't list tools I haven't run.

08 Everything else

• Google DeepMind announced Gemini 4 Carbon on October 28 and says it ranks first on version 5.0 of the Artificial Analysis Intelligence Index, at 59. For now only a subset of cyber defenders can use it. Google says it will launch at Gemini 4 Argon's introductory price of $2 per million input tokens and $10 per million output tokens.
• OpenAI is merging ChatGPT's Chat and Work tabs, starting with select users, according to an email to users that TechCrunch reported on November 2. The same email says ChatGPT and Codex will draw from one shared usage pool, so depending on usage and model, some users may wait for resets more often.

09 Second thoughts

Got it wrong. I haven't found anything in issue 008 that turned out wrong. Two of its dates are still ahead. Moonshot said Kimi K3.1 Pro's weights would be out by November 4, and Pennsylvania's AI ad label law starts December 21. If you spot an error in 008, reply and it goes here next week.

Called it. Issues 005 and 008 said to leave Mistral Large 4 alone until the weights and a license showed up. Both arrived on October 29. The license is Mistral's own, a modified MIT license that allows commercial use, though running it as a paid model service needs a separate license from Mistral. Hugging Face's ready-made deployment runs it on eight Nvidia H200 GPUs, so running it yourself is for companies with a cloud budget. For everyone else my answer hasn't changed. Use it through an API if the price beats what you have.

Checked it. Early reports said the tampered extension went out through Anthropic's official account. Anthropic says it didn't build or publish the copy, and Cursor's notice says Anthropic didn't develop it. Neither has explained how it got into the store, so I can't tell you whose account was used. Ars named one bad version, 2.1.313. Cursor's updated notice lists 2.1.312 as well. Issue 005 gave Mistral Large 4's active parameters as 52 billion from Mistral and 49 billion from Simon Willison. The model card settles it: 49 billion per token, or 52 billion counting the embedding and output layers. Every number above traces to a primary source or says who reported it. If you catch one that doesn't, reply.