We Turned Sand into Thought

Essay: How to make a token? Start with sand.

For centuries, the most brilliant minds alive committed their lives to turning lead into gold.

Newton spent more time on alchemy than on physics. They were all chasing the philosopher’s stone, a fabled substance that would transmute base matter into something precious, yet none of them ever found it. What we eventually pulled off is far more bizarre.

We turned sand into thought.

Every reply from an AI is built one token at a time. A token is a small piece of text, often just a syllable or a comma, and the atomic unit of artificial intelligence. Hundreds of billions of dollars now flow through the industry that produces them, yet almost nobody can explain how a single one is actually made.

It begins with quartz, the same mineral that ends up as beach glass, refined into the purest commercial material on Earth and grown into single crystals the size of tree trunks. The crystals are sliced into thin wafers, then etched with patterns by machines built by a single Dutch company called ASML.

ASML’s machines cost about as much as a Boeing 747. Each one fires fifty thousand droplets of molten tin per second into a vacuum and strikes each droplet twice with a laser. The second pulse heats the tin to 220,000 degrees Celsius, forty times hotter than the surface of the sun. The resulting plasma emits ultraviolet light at wavelengths found nowhere else on Earth. That light bounces through mirrors so precise that if you scaled one to the size of Germany, its largest imperfection would measure a tenth of a millimeter, before printing the chip’s pattern onto the wafer. Each finished wafer carries hundreds of billions of transistors, every one of them smaller than a virus.

The finished chips become GPUs, designed for the parallel math that neural networks require and sold by NVIDIA to data centers for roughly the price of a luxury car each. A frontier model runs on thousands of them at once, in a warehouse drawing power from its own dedicated power plant.

The model first has to learn.

Over months, it reads nearly everything humans have ever written, adjusting hundreds of billions of internal weights until it can predict the next token.

It’s only now that you can make a token.

You send the model a question, and your words become vectors, long lists of numbers that capture how each word relates to every other word the model has ever read. The vectors push through dozens of layers of matrix multiplication, each layer performing a kind of weighted vote on what should come next. After trillions of operations, the model has a probability for each of about fifty thousand possible tokens in its vocabulary. It picks one, and the whole thing finishes in about a tenth of a second.

That is everything that happens.

The model has no idea what your question means in any human sense or that it is answering it. It is an enormously expensive engine for predicting which token comes next, run at a scale where the prediction starts to resemble comprehension.

The alchemists wanted matter to become more valuable matter, but what we built is far more impressive. We took the cheapest material on Earth and made it produce tokens indistinguishable from human thought. It does this without understanding anything it says, simply by predicting accurately enough to pass for comprehension.

Newton would have called it the philosopher’s stone, finally found.

---

Thanks for joining us for our latest issue. Now go listen to our podcast :)