Subscribe | Free AI | CRO Audit | Cool Tools

Which Test Won?

If you were testing a coin to see whether heads comes up more often, how many tosses would you need to be confident in your answer? (See below.)

This week’s sponsor is Ometrics AI

News This Week

AI

Anthropic Taps SpaceX's Colossus 1 for 220,000 GPUs; Claude Rate Limits Double

Anthropic signed a compute deal with SpaceX for the full capacity of its Colossus 1 data center, which runs more than 220,000 NVIDIA GPUs. The agreement landed weeks before SpaceX's planned June IPO. Claude Code rate limits doubled overnight, and paid Claude users got expanded service. The deal sits on top of Anthropic's existing $100B AWS agreement, and reports put the company's ARR growth at 8,000% annualized. (Link)

Anthropic Dev Day: Claude Gets "Dreaming," Letting Agents Update Themselves Between Sessions

At its San Francisco developer conference, Anthropic previewed "dreaming," a research feature that lets Claude agents review their own work between sessions and update stored context like user preferences without being prompted. The company also showed off Managed Agents, a platform that shrinks agent deployment from months to days. The bigger story is the pitch for Claude as a self-improving system, which no rival has shipped publicly yet. (Link) (Link)

GPT-5.5 Instant Is Now ChatGPT's Default, Including for Free Users

OpenAI made GPT-5.5 Instant the default model for all ChatGPT users, including the free tier. The company says it brings better factual accuracy, fewer hallucinations, and personalization to everyone by default. Separately, OpenAI moved Codex to usage-based pricing, dropped ChatGPT Business annual seats from $25 to $20, and is offering $100 per new Codex user as a bounty. The combined pressure is aimed squarely at Anthropic and Google. (Link)

Meta's "Hatch" Agentic AI Heads to Testing, With Social, Shopping, and Research Built

In Meta is moving further into agentic AI with "Hatch," a consumer assistant spotted in the company's codebase and headed for internal testing by late June. It's Meta's answer to OpenAI's OpenClaw, combining image and video generation, shopping, and research, with hooks into Instagram and Facebook for social-aware workflows. No other AI agent has that kind of feed and social graph access. (Link)

Digital Marketing

1 in 5 Shoppers Has Bought From an Unknown Brand on an AI's Recommendation; Trust in AI Now Exceeds Brand Websites

A new Retail Dive report shows shoppers now trust AI recommendations over brand websites. One in five has already purchased from a brand they'd never heard of, just because an AI suggested it. What drives those recommendations isn't ad spend or brand awareness or SEO. It's product data accuracy. That's a real shift in how the top of the funnel works. (Link)

OpenAI Opens Self-Serve Ads Manager to All US Advertisers, Drops the $50K Minimum

OpenAI made its ChatGPT self-serve ads manager available to all US advertisers and removed the $50,000 minimum spend that had kept the platform restricted to larger brands. Cost-per-action bidding and third-party measurement tools are on the way. The timing is telling. OpenAI also dropped ChatGPT Business from $25 to $20 per seat. Together, the moves point to a company leaning harder on ad revenue as subscription growth gets harder. (Link)

Meta Is Preparing Personal AI Agents for All Users, Powered by Instagram and Facebook

Social Media Today reports Meta is getting ready to ship personal AI agents to regular users, not just developers. These assistants will tap a user's Facebook and Instagram activity, social graph, shopping history, and content preferences to actually personalize what they recommend.

For brands, the implication is straightforward. Your Instagram presence and product catalog become inputs to the AI deciding whether to mention you to Meta's three billion users.

Google's Personalized Search Is Here. Two People Searching the Same Query Now Get Different Results.

Product-Led SEO published a piece this week on what may be the most significant change to search in twenty years. Google's personalization engine now reaches into your entire Google footprint — Gmail, Calendar, Maps, Chrome, Search — and uses it to answer queries differently for different people.

Search "where to eat tonight" and the results might be shaped by your DoorDash receipts and Maps history. Ask a finance question and the AI might pull in your actual accountant from a calendar invite. It started in Gemini, is moving into AI Mode, and will eventually run across all of Google Search.

Keyword rankings as a measure of visibility? Pretty close to meaningless. (Link)

Google's AI Search Now Pulls Real Advice From Reddit and Forums

Google's AI Search Mode now pulls from forum discussions and Reddit threads, folding firsthand experiences into AI Overviews and AI Mode responses. It's a meaningful shift. Community content used to be a backup signal. Now it's being (Link)

Microsoft: Indexing for AI Answers Works Differently From Indexing for Search

Microsoft argues that indexing for AI-grounded answers is structurally different from traditional search indexing. Traditional search hands you a list of possible matches. Grounding commits to one answer.

Worth a read if you're trying to think through AI search visibility. (Link)

Conversion Optimization

Jakob Nielsen's 10 Usability Heuristics Are Now Free Infographics

Jakob Nielsen released a visual summary of his 10 Usability Heuristics this week as free infographics. The framework is the most-cited reference in interface design, and the new format pairs each heuristic with a design pattern that violates it. For conversion teams, it works as a friction checklist. Error prevention, recognition over recall, user control, visibility of system status. These aren't abstract UX concerns. They map directly to abandonment rates and form completion. (Link)

Samsung Treats Post-Purchase Experience as a Driver of Future Sales

CX Dive looks at how Samsung is using the post-purchase window as a revenue lever. Samsung's argument is that the days after a sale aren't a cost to minimize. They're runway for the next purchase.

Customers who get proactive setup help, product tips, and timely warranty or upgrade communications buy their next device at noticeably higher rates. If your business runs on repurchase cycles, you should probably be optimizing the days after a sale, not just the moment of sale itself. (Link)

Which Test Won Answer

Clients often ask how long a test is going to take. A few things factor in: the amount of traffic, the type of traffic and target audience, the day of the week, and the size of the difference between the variations. To cover different user behavior across the day and week, we always run a test for at least two weeks.

When I start in on statistical confidence, eyes tend to glaze over. So I pull out the coin flip analogy. If you flip a coin three times, you might get two heads and one tail. By that math, heads wins 66% of the time. Of course, the more times you flip, the closer the result gets to an actual 50/50 split.

The graph below shows the probability of heads coming out ahead. At 100 tosses, that probability is about 46%. At 10 tosses, it's 37.7%. In our actual tests, we typically have thousands of visitors and at least 100 conversions per variation, often more, and we use a few different statistical models to determine the real winner.

A couple of things to notice in the chart:

• Odd numbers of tosses always land at exactly 50%. With no possibility of a tie, heads and tails are perfectly symmetric.

• Even numbers dip slightly below 50% because a tie is possible, which steals probability from both sides. That's where the zigzag comes from.

• The oscillation dies out fast. By around 30 tosses it's basically invisible, and the probability settles at 50% from there.