The Web Was the Training Set

I read This Is for Everyone on the shore in Okinawa in the autumn of 2025 and wrote about it that winter in Looking back on 2025. That piece was about reefs and the open web. This one is about a sentence I read past the first time and cannot get out of my head a year later, after twelve months of building agent platforms for a living. It is the last line of the last chapter: “AI had aced the Imitation Game, and the web provided the data that made it possible.”

Berners-Lee writes it as a coda. I think it is the thesis of the book, and I think he undersells it.

What the book is

Three books share the binding. The first is a memoir: a boy in East Sheen whose parents wrote code for the Ferranti Mark 1 and knew Alan Turing, physics at Oxford, a homebrew computer built from factory-reject transistors and an adding machine fished out of a skip at a timber yard, a first job at Plessey in Poole. The second is the history everyone buys the book for: CERN, the 1989 memo, the NeXT cube, the first server, Mosaic, the World Wide Web Consortium. The third is a manifesto for the next thirty years: data sovereignty, Solid pods, and an AI that works for you rather than for the company that trained it. Stephen Witt, who wrote The Thinking Machine about Nvidia, is the co-author, and the prose is clean. The structure is looser. The book proceeds by association, which suits its subject, and which is also why some reviewers found it shapeless.

Compressed into one argument: the web won because it was universal and free, both qualities are now being eroded from the inside, and the way out is to give people control of their own data so that the next layer, the agents, works for them. I think he would accept that summary.

The epicentre was not where you expect

The book is strongest as geography. The web did not come out of Silicon Valley, and Berners-Lee is careful to say he barely knew the place existed when he graduated. In 1977, he writes, no Silicon Valley recruiter came to Oxford. His computing education came from his parents, from Manchester by way of Ferranti, and from the strange luck of being born in 1955, the year after Turing died, into a house with the Mark 1’s punch tape on the floor.

Switzerland comes before CERN. His first job abroad was in Zug, writing microprocessor code for a wafer saw, and he dates his attachment to the country to a night of Kaffee Fertig in a chalet above the cows: “I think it was then that I first fell in love with Switzerland. I’m still in love with it today.” CERN follows, first as a six-month contract in 1980 and then as a fellowship from 1984, with an apartment in Saint-Genis across the French border and windsurfing on the lake. The web was conceived in that corridor, at the coffee counter where people from twenty countries worked out how to run a machine nobody fully understood. The 1989 memo says so in its first argument: “CERN is a model in miniature of the rest of the world in a few years’ time. CERN meets now some problems which the rest of the world will have to face soon.”

What I had forgotten is how precarious the whole thing was. The question that hung over the project, in his own phrasing, was “Why, exactly, is the European taxpayer financing the oddball passion project of a thirty-six-year-old computer programmer in the Data and Documents division of a Franco-Swiss particle physics laboratory?” The answer, for three years, was that two managers, Mike Sendall and Peggie Rimmer, chose to hide him from the budget. Sendall’s pencilled “vague, but exciting” on the memo is famous. The chapter around it, about Sendall’s illness and the marriage Peggie entered to be able to care for him in a Swiss hospital, is the most human thing in the book, and it is a reminder that infrastructure gets built because particular people protect it for a while.

The British bookend is the 2012 Olympics, where Danny Boyle asked him to type the title phrase on a NeXT in front of 80,000 people as one of the inventions “Britons had shared with the world.” Between Geneva and London there is the one scene I would put in every history of computing: the 1992 NeXT developer conference in Paris, where Berners-Lee set up the web on a folding table to show Steve Jobs, and an aide pulled Jobs away one table early. He calls it a sliding-doors moment. I think he is right, and I think the door that closed was a European one. Two years later the consortium went to MIT, with a foothold at INRIA. The centre of gravity followed the money.

Free, because Gopher was not

The chapter I would make mandatory reading for anyone who writes policy about open standards is the one about Gopher. Until 1994 more people used Gopher than the web. In February 1993 the University of Minnesota announced that it reserved the right to charge licensees. It never collected a cent, by his account, and the announcement alone killed the protocol within months.

Berners-Lee’s reaction is the part that matters. He realised the web was in the same position: written on CERN machines, on a CERN salary, owned by CERN. “For the web to succeed, it had to be free.” Inside the lab there was a faction, he calls them “patriots”, who wanted the opposite: spin the web out as a company “to prove that Europe could compete” with Silicon Valley. He was sympathetic and overruled them. On 30 April 1993 CERN’s directors signed a statement that “CERN relinquishes all intellectual property rights to this code, both source and binary and permission is given to anyone to use, duplicate, modify and distribute it.”

I agree with the decision and I want to be exact about what it cost. Europe funded the research, hosted the invention, and gave away the result without keeping an institution that could compound on it. The protocol stayed free; the platforms built on top of it did not, and none of the large ones are European. That pattern, the commons funded here and captured there, is the background to the argument I made in The Bottleneck Is Data, and Europe Is Sitting on It, and reading this chapter again made me more convinced of it, not less. The public-domain release was correct. The absence of any European plan for what came after was the mistake, and it was not his to make.

The web was the training set

Now the sentence. Berners-Lee’s own account of machine learning is short and, for a man who spent two decades on the semantic web, generous. Machine learning, he writes, “has three legs: software, hardware and data,” and “it is the third leg, the data, that is in many cases the missing piece of the puzzle.” He tells the GPT story with the detail I had not registered: the first GPT trained on a corpus of self-published romance and science fiction, and “its output was closer to nonsense than meaningful human prose.” The models became useful only once OpenAI turned to the non-profit archivists at Common Crawl and trained on the web itself. His conclusion, in the chapter on intelligent machines: “Data was the fuel for this breakthrough” and, specifically, free and open web data.

The numbers back him. In the GPT-3 paper, filtered Common Crawl is 410 billion tokens and 60 per cent of the training mix by weight; the rest is more web (WebText2), two book corpora, and Wikipedia, which is also the web. The 2024 PNAS study he cites as the Turing Test being passed found that a GPT-4 model’s choices in behavioural games were statistically indistinguishable from a random person drawn from tens of thousands of human subjects in more than fifty countries. Whatever one thinks of the framing, the mechanism is not in dispute. The models learned to write from what people had published, and people had published it because of the thing this book describes.

What I want to add is why the web worked as a corpus, because the reasons are design decisions in chapters 2 and 3, not accidents. A page is a document with a stable address. Text is marked up, so titles and headings and lists can be told apart. Links are explicit, so relevance can be computed; PageRank is a graph algorithm over anchor tags, and Common Crawl’s filters and deduplication lean on the same structure. The hash sign, the place where in his words networking and hypertext “meet”, is also where a crawler learns which part of an address is a document and which is a position inside it. He describes the five-star ladder for open government data later in the book: on the web, structured, non-proprietary, addressable, linked. The web as a whole climbed that ladder without being asked, and a quarter of a century of people exposing their writing to each other in a structured, addressable form is what a transformer was eventually pointed at. The commons was the dataset before anyone called it one.

Two consequences follow, and the book reaches the first but not the second.

The first is his. The same openness that let the web grow is why the text could be taken without asking. He is blunt that Common Crawl “contains a lot of copyrighted material” and that the models can reproduce it, and he lands on a position I did not expect from the man who refused to patent HTTP: “An AI should only have the right to use your creative work as input if you have explicitly opted in, and should you opt in, you have a right to get paid.” He proposes a Content ID for text and admits the attribution problem is unsolved, with a GPT-4 ode to a dishwasher in the style of Philip Larkin as the test case. Nothing in it is Larkin’s, and all of it is. This site, for what it is worth, publishes a Content-Signal line in its robots.txt and a llms.txt and asks for attribution, which is roughly his opt-in with the terms attached. Whether anyone reads it is a separate question.

The second is mine. If the web was the training set, then what was never put on the web is not in the models. That is the whole of the European data argument in one line. The archives, the press runs, the parliamentary records, the century of paywalled newspaper pages, the scientific proceedings in twenty-four languages, exist in machine-readable form or could, and sit outside the corpus the frontier models were built from. Berners-Lee’s own life makes the point: the data at CERN only became valuable to physics once his system put it where other people could find it, and the same is true of the data Europe is sitting on now. The next increment of model quality is not in a larger crawl of the same web. It is in the material the web never got.

Where I disagree with him is on the mechanism of intelligence. He argues that scale settles the question: feed a model “a high proportion of all the sentences in the whole world” and “it builds a network with the number of nodes comparable to the number of neurons in your brain.” Parameters are not neurons and the comparison is loose; I read it as enthusiasm, not as a claim he would defend at a whiteboard. He also makes the stronger philosophical claim that there is no difference between simulating consciousness and having it. He is entitled to that view, and I would have liked a chapter on it rather than a paragraph.

Engineering lore

The chapters on the first year of the web are the most fun for anyone who ships software, and they are honest about regret in a way founders rarely are. The URL, in his account, was “the most important thing I ever wrote”, a few lines of code that reduced the friction of moving between machines to a click. The rule for the whole design was to reuse what people already knew: SGML for the markup, DNS for the names, Apollo workstation paths for the addressing. The double slash came from Apollo, and he apologises for it. The DNS order came from the United States, and he wishes he had used the British convention, most general part first, so that addresses read uk.co.widgets.www. The read-write browser he built on the NeXT, where every page was editable, lost to read-only browsers because users preferred one tool for reading and another for writing, and the web became “a lot more passive” than he intended. Marc Andreessen added the IMG tag to Mosaic unilaterally while the argument about a more general design was still running, which is exactly what a public-domain standard permits. The whole first browser, editor and all, was 9,555 lines of Objective-C.

There is a sentence in there for people who build platforms: “software designers, like urban planners, make the mistake of demanding that the user acts like the system, rather than the other way around.” The system behind the web is rigidly modernist, he says; the web on top of it was designed to be an anarchic jumble. Most of the platforms I have seen fail did so by letting the first layer leak into the second.

Charlie, and who the agent works for

The last third of the book is about Solid, the protocol for personal data pods that Berners-Lee has been building since the semantic web years, and about Inrupt, the company he co-founded to commercialise it. The argument runs through a 2017 dialogue with an imagined assistant called Charlie. Charlie reads a chef’s cookbooks as a training set to reconstruct a dish, invests in a start-up to give Bob something in common with his mother-in-law, and when asked who it works for answers: “Legally, ethically and algorithmically, I work for you, Bob.” It was written before ChatGPT, and it reads now like a product spec for the agents everyone is building.

As someone who builds the other side of that spec, I think he has the problem right and the solution only half right. The problem is real: the apps on a phone do not share data because they do not trust each other and do not speak the same language, and the agent that gets access to everything is the one that most needs to be trustworthy. That is the point of a context layer, which I have argued is the product and a write path, not a cache. Pods are a reasonable place to put that layer. They say nothing about who runs the model that reads it, where the inference happens, or what the agent is allowed to do with what it reads, and that is where the agent platforms I work on spend most of their effort. A pod makes the data yours. It does not make the agent yours.

The Solid chapters also read like a pitch, which they are, and Publishers Weekly’s complaint that the scheme “sounds fuzzy and dull” in his telling is fair. The ideas are not dull. The telling is, because it is written from inside the company. I would have cut a third of it.

Verdict

I give it 8 out of 10. The history is first-rate and told by the one person who can tell it, with the Swiss and British years given their due and the Gopher chapter alone worth the price. The AI chapters contain the most important idea in the book, that the web is the reason the models exist, and then leave it to the reader to work out what follows. The Solid chapters are a prospectus. The structure wanders, and the forecast is thin next to the history; John Arlidge in The Times called it 400 pages with no driving narrative, which is harsh and not entirely wrong.

Read it if you build anything on the web or on models trained from it, and read it for the history first. You do not need a technical background; the hardest idea in the book is the hash sign, and he explains it in a paragraph. Pair it with Mallaby’s The Infinity Machine, which I am reading now and which picks up the Hassabis thread this book opens at a Google dinner in 2014. And read the last line again when you are done. He provided the data. The question of what we do with the rest of it is still open, and it is a European question more than he lets on.

References

• Tim Berners-Lee with Stephen Witt, This Is for Everyone (Macmillan, 2025). Quotations are from chapters 2, 3, 4, 10, 11, 12, 16 and 18 of the UK edition.
• CERN, statement of 30 April 1993 releasing the web software into the public domain. CERN’s account, Twenty years of a free, open web, includes Robert Cailliau’s recollection of the six months of legal work behind the one-page document.
• Tom B. Brown et al., Language Models are Few-Shot Learners (2020), arXiv:2005.14165. Table 2.2 gives the GPT-3 training mix: filtered Common Crawl, 410 billion tokens, 60 per cent by weight.
• Qiaozhu Mei, Yutong Xie, Walter Yuan and Matthew O. Jackson, A Turing test of whether AI chatbots are behaviorally similar to humans, PNAS 121 (9), 2024. The study the book cites as the Turing Test being passed.
• Common Crawl: monthly crawls of roughly two billion pages each; the cumulative archive is commonly cited at more than 250 billion pages, well above the figure in the book. See also my Europe data piece for the Epoch AI estimate of the public text stock and the FineWeb yield after filtering.
• Sebastian Mallaby, The Infinity Machine: Demis Hassabis, DeepMind, and the Quest for Superintelligence (Penguin, 2026).
• Other reviews: Publishers Weekly; Tara Shears, Physics World; John Arlidge, The Times, September 2025; James Gleick, How the Web Was Lost, New York Review of Books, 11 November 2025.