How the Web Spread When No One Knew What It Was

It’s 1991. Tim Berners-Lee comes into work, opens up his computer, that very same NeXT machine where the code for the World Wide Web originated just a year earlier, and checks his email. It’s a routine most of us have. He’s looking for something specific though and, after not too long, he finds it. Overnight and across the globe, someone else had setup their website. It’s nothing necessarily extraordinary, another university’s computer science department must have gotten curious about the web, tinkered around for a night, and gotten something up. As usual, someone had thought to post to the web’s mailing list to let people know about it.

Berners-Lee clicks the link and is brought to a bare, unembellished page. Behind the scenes it’s little more than a small chunk of HTML, the markup language he had created specifically for web browsers. The site itself is buried on some subdomain on the university’s server. It has a few blocks of text, a directory of students and faculty in the department, and a few links to some other interesting websites. Just below that, there’s a brief description of the World Wide Web project. Sometimes, in the early days, websites did that. They explained the web on the web for those that had somehow managed to find it without knowing anything about the web.

He pokes around this new canvas for a minute, then does what he always does when someone sends him a new website. He pops open his editor, and adds it to a list of links he keeps on the CERN site called the WWW Virtual Library. That list is still very small, but this latest link brings the total number into the dozens. Pretty soon, he’s going to have organize all of these links into better categories. Deep down, he knows there will be a lot more soon.

The WWW Virtual Library, from Tim Berners-lee

How did the web spread? Little by little, person to person. On mailing lists and in hallway conversations at conferences and on BBS boards and elsewhere in what was already a great digital abyss. The web enabled computers to connect to other computers but it is, and has always been, people all the way down. People loved the web, so people spread the web.

But it wasn’t always that way.

Staring Down Behemoths

In 1985, the fairly little known modem-as-a-service company Control Video Corporation, unable to get much traction in an already crowded market, was forced to close up shop. From its ashes rose a new online services startup, Quantum Computer Service, founded by executives from CVC. A few years later, that same company would rebrand as America Online and introduce the American public to the network.

Led by the charismatic and often bullish CEO Steve Case, and backed by an aggressive marketing strategy that, as I’m sure at least some remember, included sending millions of unsolicited free-trial floppies (and later CD’s) to households around the U.S., AOL spread across America in a few short years. By the mid-90’s they were pushing past established competitors that had been leaders in in the industry for a decade. CompuServe and GENie were among the first online service providers in the early 80’s, and their loyal techie fan base was enough to help spread into larger markets. Prodigy formed several years later as a partnership between IBM, CBS, and Sears, leveraging their deep pockets and sizable investment to seize control of the market. Now, in just a couple of years, the upstart AOL was more popular than all three.

Online service providers, like AOL and Prodigy, offered a simple package to its customers. A connection to an isolated network and a user interface for cruising around that network, typically with a sliding scale, depending on the provider, of attractive features layered on top like email or private messaging or digital content from editorial providers. Think of them as a friendly door carved into the side of a network. Once you walked through, you could talk to friends, or go shopping, or read the news, or explore an entire digital world created by your provider and tailored just for you. The catch was, in stark contrast to the web, if you wanted to visit another digital world from a different online service provider, you had to walk out one door and through a different one (a lesson that has not been lost on modern social networks which are largely attempting to replicate the walled-in, all-in-one experience of early online services for the sake of audience ownership and profitability and at the expense of personal privacy and distributed networks, a point that Alexis Madrigal made years ago. But I digress). Each was tethered to its own infrastructure and completely isolated. Growth at an exponential rate was key.

AOL’s meteoric rise owed its fair share of credit to shifting awareness of the fact that online wasn’t just something you did at the supermarket. More and more people were learning each day what online networks were. AOL was the friendly, approachable outlier of the bunch. Access to networked communities had largely been confined to the technologically adept, those that felt comfortable enough with computers to at least set up their own connection. Their customers were made up of people that might, at the time, proudly label themselves as geeks, or if they were feeling a bit prideful, digital pioneers.

AOL was after the other type of people, the cyber curious, the parents and friends and colleagues who had heard about a magical world “online” but didn’t know where to start. AOL offered a lifeline, an easy to install and setup online package, with a free trial just in case you were on the fence. Their network was massive, and dizzyingly vast in its scope. They had partnerships with MTV and the New York Times. They added instant messaging for one-to-one chat. They hosted chat rooms and message boards about pop culture and news and parenting, topics that ranged far beyond the tech-speak of other providers. As Mark Nollinger put it in a 1995 expose of AOL,

You want “information”? Fine, check out CompuServe. But if you want to swap opinions on Medicare cuts or hang out with Courtney Love online, AOL’s the place to go.

AOL had captured the digital zeitgeist of the early 1990’s. Journalists wrote about their overnight success, with some anticipating as a foregone conclusion complete control of the market in just a few more years. Profiles of their illustrious CEO graced the cover of popular magazines. In meetings across Silicon Valley, startup founders were fed the story of their success as a playbook. Eager investors, frustrated at missing out on the ground floor of the next big thing, took meetings with promising copycats. They were a household name, the centerpiece of dinner conversations. The future, it seemed, had arrived, and its name was AOL.

The infamous AOL loading screen
A familiar screen for many of us I’d imagine

Off to the side a bit, there were a few independent efforts waiting in the wings, mostly experimenting with the still primarily academic and governmental network known as the Internet. Brewster Kahle, already making a name for himself at the supercomputing firm Thinking Machines shortly after his graduation, was working on a way to tie Internet servers together, which he was calling WAIS. It was a fairly straightforward way for people to store documents on the Internet, coupled with a powerful user interface for searching through and downloading those documents across a decentralized network. Over at University of Minnesota, a few programmers had been toying with a similar idea, a sort of Internet directory for storing, organizing, and locating content and files online, backed by rudimentary categorization and search. They playfully named it Gopher, half as a nod to their own college’s mascot, and half to illicit the feeling of burrowing through the Internet.

These experiments, however, were mostly confined to theoretical tinkering. They were too technical for the average consumer. There were plenty of imagined practical applications going around in insider tech circles, but they felt a long way off. It was enough to give some (who were paying attention) pause. Perhaps there was competition outside of closed online networks. It was unclear what that might look like, but there was quite a possibly a there there.

A far more clear and present threat to the dominance of AOL was the almost mystical and ultimately failed service planned for release by Microsoft. Known only as The Microsoft Network (MSN), it promised a lot, all the bells and whistles of a modern online service service plus the added bonus of being bolted right on to the most popular operating system in the world. Microsoft was as close-lipped as ever about the project, so the rumor mill produced a number of wild and imaginative features for the platform. It was going to be a one-stop shop for online services. It would have an AOL-killer chat. Some even thought it might be free for Microsoft users. If you asked Steve Case about it, and plenty did at the time, he’d decry MSN – which I’ll remind you was yet to be released – as an unfair assertion of their market position and a strict violation of antitrust legislation in the United States. Microsoft couldn’t be allowed to own both the software on computers and the software on the network. It is perhaps unsurprising then, that several years later, when Microsoft was brought up on antitrust charges for an unrelated but a still analogous push into the browser market with their free release of Internet Explorer, Steve Case was among those who testified against Bill Gates and Microsoft for strong arming the industry.

Anywhere and everywhere in the tech world people were talking about the impending conflict between these two massive corporations. Backed by user bases that numbered in the millions, many predicted that when the clash between Microsoft and AOL finally did happen, the declared winner would own networked communication for years and years to come.

All of which is to highlight one fundamental point about the early ’90’s. There was a lot of buzz about online networks, and what they could mean in the 21st century. In the right circles, word of “the network,” and who would own it, was everywhere. And in all of these conversations the World Wide Web was barely ever mentioned. It wasn’t that people thought the web couldn’t win against industry behemoths like Microsoft and AOL. It’s that nobody even knew they were playing.

Spreading

One of the biggest problems of selling someone on the web in 1991 was that it made very little sense until a critical mass of people were using it. If you were to say, boot it up on a single computer and demo it in the corner of a hypertext conference – as a not so hypothetical example – it would look far from impressive. To remedy that particular problem, and to help spread the web outside of the hallways at CERN, Tim Berners-Lee wisely recruited a small team of people from inside it. They would make up his original team, and were responsible for the very first iterations of the web outside of a single computer in the office of its creator.

Nicola Pellow, an intern programmer at CERN, was the first to tackle the web’s biggest technical problem. Berners-Lee’s browser only worked on NeXT computers, which were expensive and in short supply in the market. She developed the line-mode browser, a text-only browser that nevertheless had the useful advantage of being instantaneously recognizable to most programmers and compatible with every operating system and platform. Without the line-mode browser, the web would be relegated to a select group of people that actually bought a NeXT computer with its massive processing power and beautiful UI; a group that, suffice it to say, was not enough for any kind of forward progress. Pellow and Berners-Lee also saw fit to release the code for the browser to the larger community, so others could build from it.

Nicola Pellow’s text-only Line Mode Browser

Robert Calliau, another colleague at CERN, and the first true convert to the World Wide Web, took on the extremely hard job of keeping the project alive. He lobbied relentlessly inside of CERN, ensuring the skunk works project had what it needed in terms of funding and office space and travel expenses. CERN agreed to back Berners-Lee and his pretty-out-there hypertext software, as long, of course, as people were actually using it. Calliau made sure that happened. He traveled with Berners-Lee to conferences to demo and spread the word about the web, and a couple of years later, he’d organize the web’s first conference of its own.

The team was small, but fiercely passionate about the web project, and what it could do for the world. They recognized that in its simplicity and fragmentation there was an odd and unforeseeable power. It put the means of publishing into the hands of everybody, and spread information out in every direction. They could see their vision right around the corner, starting to form, become more clear with each passing day. One day, Berners-Lee decided the web was ready to be shared. He posted a link to download its source code on a Usenet group, so others could experiment with it. Some technically curious individuals did just that, and a small group of people began to gravitate toward the project.

In 1991, Berners-Lee created the www-talk mailing list, one that continues to this day, so these early web adopters could have a place to swap ideas and surface bugs and generally talk about the World Wide Web’s future. A year later, much of that conversation moved over to the Usenet newsgroup comp.infosystems.www. Both are extensively archived, and as relics of digital history, provide an incredible window into the shifting attitudes and dynamic conversations that surrounded the early web.

Programmer Pei-Yuan Wei hopped on the mailing list late ’91 to share a bit of code, somewhat intrigued by the promise and potential of the web. Tinkering after hours at his job at UC Berkeley, he had ported a bit of his own scripting language to read and render HTML. In other words, in a single night, Wei had cobbled together the very first browser built by someone that wasn’t an employee at CERN. It wasn’t exactly cutting-edge, not yet at least, but it worked on X Windows machines. Pretty soon, Wei would find himself among the earliest web pioneers, and even later, a founding employee at Global Network Navigator. That little bit of code he wrote would transform into a popular choice among early web users. It would be the template and standard by which the first generation of browsers were measure against. But in his post on the mailing list at the end of 1991, it was a hacked together idea Pei Wei was looking to get some feedback on. He was calling it ViolaWWW. Later on, most would simply refer to it as the Viola browser.

David Farley had a different kind of experiment he wanted to share. By day, he worked as a staff member at the library of the University of Chicago. In his spare time though, he liked to draw one-panel comics, a blend of offbeat humor and an expressive drawing style, filled with talking animals and ludicrous situations. A friend set him up with a small bit of space on the University of Chicago’s server so he could post some of his comics to a website. When Farley first posted a link to his Doctor Fun comic on the web usenet group in ’93, he didn’t even realize it was the first webcomic. A couple of weeks later, the folks at Mosaic chose to highlight it on their once-daily updated list of links “What’s New.” They called it a breakthrough. They weren’t wrong. For over a decade, Farley would continue to update Doctor Fun every weekday with a brand new panel.

Some found the web a bit dizzying, even when the number of websites still numbered in the hundreds, rather than the billions we have today. The web was, by design, unrestricted. A bit of technical know-how could get you a website, and anyone link to anywhere, even websites that no longer existed (or never did), leaving links to rot on the vine. It’s freedom was, at times, a burden for its users. Some found discovery a struggle.

Oscar Nierstrasz was among the first to sympathize with the plight of the confused web explorer. He pieced together a small script that compiled a few “best of” lists into a single place. The W3 Catalog, as it was called, was updated each day with the very best links from around the web. Nierstrasz even spun a bit of a sales pitch when it posted it to the comp.infosystems.www Usenet group:

Tired of searching for exciting pages with Xmosaic? Don’t have access to a robot? Why not check out what you colleagues think are interesting pages?

Little conversations like this, ironically tinged sales pitches and geeky back and forth about a brave new world was exactly how things got started. The conversation that followed this thread was largely about the legality of crawling websites programatically. In the beginning, this was how things were figured out. They were hashed out in long running threads that web devotees tended to in  their spare time.

Over in California, or Texas, or France maybe, somebody gets a web server running for their CS department, and shares their code. A few others discuss the limitations and possible expansions of HTML to introduce design and presentational elements to webpages. Everybody was trying to figure out what the web could be, and they were doing it out in the open.

Then Jim Clark walked into the offices of NCSA, walked out with a group of engineers, and strapped the web to a rocket ship.

Mosaic Communications Corp

This is a story that’s been reported a number of times, told and retold in interviews and pitch meetings and, more recently, in a limited television series. It has, at this point, entered Silicon Valley mythology, alongside the ousting of Steve Jobs and the Hewlett-Packard garage. In an effort to not overstate the importance of or dive too deep into what has, at this point, become well-trodden territory, this a a mostly abridged and fairly concise version of the aforementioned story, which goes a little bit like this.

Eric Bina and Marc Andreessen liked the web a lot. They worked at the National Center for Supercomputing Applications (NCSA), a research lab at the University of Illinois fresh off a recent round of funding from the Gore bill, the legislation behind the sometimes mocked but also kind of true “I invented the Internet” line by former Vice President Al Gore. Their jobs had nothing to do with the web – much like Berners-Lee’s work at CERN – but they found a bit of spare time to work on a browser anyway. Bina and Andreessen believed that the web lacked a friendly face, that the browsers available were difficult to install, clunky and hard to use.

Together they built Mosaic, a browser backed by a sleek interface, inline graphics and interoperability. Within a few months of its first release in early ’93, it was the only recommendation anybody seemed to have when somebody hopped on the mailing list to ask for one. They even got a bit of press. Pretty soon, everyone at NCSA wanted to stop whatever it was they were doing to work on Mosaic. They took turns adding new features, or porting the code to different operating systems. Andreessen even had the idea to toss up a “What’s New” page whenever the browser was opened. It was updated once a day with the newest and coolest sites. Mosaic felt different than other browsers. It was edgier, it had color and animations and every day it seemed there was something new for developers to play around with. More than that, it demonstrated what the web could be, and that it could be for everybody.

Then, Andreessen left NCSA. He had been getting a bit restless, and hoped to push west to the great frontier of Silicon Valley. Which just so happened to be around the time that entrepreneur and tech giant Jim Clark was getting a bit restless himself after spending a decade at Silicon Graphics, Inc., a company he had helped start. Someone gave him Andreessen’ name, and he sent him an email. It read:

Marc:

You may not know me, but I’m the founder and former chairman of Silicon Graphics. As you may have read in the press lately, I’m leaving SGI. I plan to form a new company. I would like to discuss the possibility of your joining me.

Jim Clark.

Andreessen was, of course, plenty excited at the prospect of working with a tech giant like Clark, so the two of them met to put together something more concrete. They tossed around a few ideas, none of which had anything to do with the web. If they had stuck with one of those ideas, we might have a very different World Wide Web. Instead, Andresssen casually tossed out different thought. What if they just created a better version of Mosaic? Clark agreed. But if Andreessen was going to build a new browser, and do it right, he was going to need his team.

The next week, Clark and Andreessen hopped on a plan and headed to Illinois. They held a secret meeting in the lobby of a hotel with a few members of the NCSA team to reveal their big idea. A Mosaic killer. A browser that was better, sleeker, easy to install, and aimed at average consumers. One by one, Andreessen filled their heads with the promise of a bold future. Clark sweetened the deal with a generous package to abscond with them to the Valley. Every member in attendance accepted that offer, and the next day, 5 NCSA employees quit and moved out west to California to form Mosaic Communications Corp.

About 6 months later, towards the end of 1994, they released the first version of Mosaic Netscape, later called Netscape Navigator. Andreessen had delivered on his promise. It was unlike anything the web had seen before. It had all the great features of its predecessor but, for both legal and logistic reasons, it was rewritten from the ground up. It was more polished, more refined, it had bookmarks and layouts and a browser bar with handy links. It was the very first version of what we think of as a browser today, a concept that, until Netscape’s release, was still very much an open question.

Netscape Navigator: The world’s first commercial browser

Netscape closed the book on that question. For better or worse, in the mid-90’s, most people figured that Netscape was the web, it was the only browser they had ever heard of, and the only way they had ever used the web. In just a few days, it was downloaded tens of thousands of times. In a few weeks, it would overtake the original Mosaic browser as the most popular browser in the market. In sold in stores, it was shipped with Internet modems. They were picked up in the press, and not just by esoteric computer magazines, but by the New York Times and Wired. A year later, they would hold the most successful IPO in tech industry. Netscape became the web, but it many ways, it enabled its success. It provided the last, missing piece. It gave the web to everybody, and that, that was always the goal.

What A Difference a Year Makes

In an article for Inc magazine in 1996, journalist Alessandra Bianchi centers her premise about Netscape’s control of the web on a particular moment on the campus of MIT in the spring of that year. Sometime in late May, Bill Gates gave a talk in the famous Kresge Auditorium to a packed room of students, faculty, and a completely filled press box. It had been the buzz around campus for weeks. Just a few months earlier, Jim Clark had come to that very same auditorium, similarly packed wall to wall, to speak about the massive influence of Netscape, fresh off their massively successful IPO. In stark contrast to both, a few hours after Gates’ talk, Tim Berners-Lee took the stage to a much more modest audience of web devotees and a few scattered members of the press. He was there to talk about the W3C, a standards organization starting to gain a bit of traction but nevertheless almost completely unknown – as it remains to this day – to the larger web public.

Bissichi isolates this moment to frame a much larger discussion about the battle for control of the future of the web being waged mostly between Microsoft and Netscape. Ultimate control of a decentralized global network by a handful of mega tech corporations, Bissichi argues, is far from an ideal situation. Berners-Lee was offering a more universally accessible alternative in the form of the W3C. A way of reigning in control of the web and bringing lots of different voices to the table.

Yet Bissichi’s short anecdote, over 20 years later, clarifies an even different point. Jim Clark and Bill Gates, two of the tech industries most powerful executives, took the stage to talk about the web. They could’ve talked about any technology, but the web was what was getting them excited. Most people in the room didn’t even know who Tim Berners-Lee was, the web’s earliest pioneers had already been almost completely forgotten. But everybody knew what the web was. It had finally spread, beyond the hallways CERN, up through the hidden-from-view mailing lists and out into the world.

Even more compelling is that this massive shift from obscurity to ubiquity happened in essentially a single year, somewhere in between the time of Netscape’s official public release and the beginning of the browser wars. In 1994, nobody knew what the web was. In 1996, it was all anybody could take about. So if you ask me for a turning point, I can actually point you to one, or at the very least, the first one. It’s 1995.

More on that next time.

Sources