The History of the Web

There’s a website developed with a personalized experience in mind. It touts major breakthroughs in predictive technology, boosted by user generated content and driven by sophisticated algorithms that provide real-time recommendations.

The long term future promises a personal assistant that can follow your decisions and choices across the webs landscape and help you find what you’re looking for before you even know you need it.

Of course, there are concerns with the site. It requires an unsettling amount of access to personal information, and it’s unclear if it will deliver enough value to warrant it. When it works, it works well. But when it doesn’t, it can prove to be an inaccurate nuisance, and all the magic is lost.

A website like that feels perfectly normal in 2024. But a website like that actually did once exist. It was called Firefly. And it was created and launched all the way back in 1995.

Firefly gives us a roadmap for thinking about generative and predictive technologies today, specifically AI. All of the same cycles, all of the same conversations we’re seeing today, were played out over the course of a few years right before the turn of the century. And from the day it launched, it’s been a template for how sites get built in the Web 2.0 era, up through today.

Firefly’s Beginning

Firefly’s first major product in 1995 was called Bignote, a website where users could rank what kind of music they liked and be given a list of personalized recommendations. The more you used it, and added your musical taste to it, the better the recommendations got.

Bignote made use of a concept known as collaborative filtering—something akin to what we call an algorithm today—for its site. The insight behind collaborative filtering is that people’s tastes, it turns out, aren’t random or evenly distributed. “There are general trends and patterns within the taste of a person and as well as between groups of people.” In other words, if you’re into something that somebody else is, you can probably agree on lots of other things too. And if you get enough people to tell you what they like, you can use that correlation to get pretty good at predicting what they might like next.

Like so many things in the computing era, one of the earliest experiments with collaborative filtering happened at Xerox PARC, in an early experiment that filtered content from newsgroups. It was then taken up by the MIT Meida Lab by Pattie Maes, who developed an early prototype of Bignote called Ringo with her colleagues and students. Some of those students eventually created a commercial version of Ringo, and that’s how Firefly was born.

Firefly depended on data, and they were one of the first companies to realize that people actually like talking about their personal taste and stacking it up against others. Music, something people have strong opinions about, worked the best because all it took was a little push, and visitors were happy to hand over their opinions. Just giving people a nifty interface and a rudimentary rating system was all they needed to start collecting a good amount of data.

It helped that their interface was actually pretty good. Years before AJAX and DHTML powered web pages gave websites like Delicious and Gmail the feeling of using a genuine desktop application, Firefly managed to create a user experience that was quick, easy, and fun. They even built a really early version of instant messaging right into their platform.

“How refreshing, then, to come across a site that we can almost ungrudgingly admit ‘gets it,'” an early review at Suck had to admit, “Deeply, at that.”

Firefly’s major conceit was that the web enabled a global scale of data collection that previously been impossible. And that made true collaborative filtering possible for the first time. Pair that with some actual value for users in the form of recommendations, and a targeted map of a user’s particular interests for advertisers, and it’s not hard to see how Firefly became a hyped bit of technology on the burgeoning commercial web.

Firefly Commercial Success

After Bignote, Firefly began to pick up some speed. It introduced some interesting community elements, with a built in chat and personalized homepages inspired by a still-new Geocities.

It got really interesting when it was able to pair you with others that had similar interests to you, “trusted neighbors” as the site referred to them as. Being able to create little communities of like-minded folks on the site hinted at what the site eventually anted to become.

Firefly had ambitions to move well beyond just music. Bignote, and other early experiments, were just a way to get you in the door. Firefly was meant to integrate into different services and stores, bookstores like Barnes and Noble, travel sites like Expedia, directories like Yahoo!, and help gather your tastes and preferences along the way.

The platform eventually evolved into what they called Firefly Passport, a more generalized personalization algorithm that could shape your tastes as you interacted across the web.

Writing in 1997, a news story painted the picture of what a finished Passport application would look like.

Fast-forward two or three years into the future. Firefly (the company) may not have survived its battles with competitors, but software agents will not only have survived, but also be more powerful. As soon as you give your agent your personal tastes about restaurants, plays or clothing, it begins fetching things for you — even things you didn’t know you wanted. Your agent knows that you like Latin jazz, for example. A record company with access to the data sends you an ad: a Tito Puente CD is about to come out. Knowing that music and food tastes sometime overlap, your software agent even recommends that you try a new Caribbean restaurant in your neighborhood.

Firefly wasn’t just a place to find cool new music. It was your everything algorithm, directly integrated into all of your favorite websites. A computerized, personal assistant that was always there, always watching, learning more and more about you. But in 1997, they were calling that an agent.

Concerns About Firefly

By the time Firefly released the first version of Bignote, competition was already mounting. Agents had become the center of the web’s latest hype cycle, the last one before the rise of the dot-coms. Other scrappy startups hoping to carve out niche were soon joined by the big players; IBM and Microsoft each started experimenting with personal agents.

What exactly agents were supposed to do was a little vague. In some imagined future, once they had consumed enough personal information and patterns, it was said, agents were a personal personalization engine. By gathering preferences over time, web agents could make suggestions about what you might like to listen to, or to watch, where do you go out to eat or what to do with your day. Something like an Amazon “You Might Also Like” bar, tailored to your tastes.

In fact, Firefly’s original name was Agents, Inc. In 1997, they changed their name and went by Firefly instead. The mission was the same. But the public was getting unsettled about the idea of a magic robot that followed them around the web. It all felt a bit… invasive.

The concern was in how much of one’s personal data needed to be revealed to some magical black box, and whether or not the tradeoff was even worth it when the actual results were purely hypothetical. Personal agents had access to everything you did. Before tracking people around the web was something commonplace and banal, that felt a bit too much like spying.

Microsoft evaded these questions thanks to their vertical integration. IBM tried to pass off liability to their licensing partners. In public profiles, Firefly mostly hand-waved these concerns away. They did, however, begin to work with other companies to create privacy standards for their software. And over time, they tried to distance themselves from the concept of personal agents altogether, preferring to use the term “personalization” instead.

What Eventually Happened

After the initial hype wore off, Firefly had trouble selling its application. It was all a bit too hypothetical, and couldn’t grab a big enough user base. After a year or two of trying to sell the software to consumers and big name brands, the app faded to the background, though it continued to power the recommendations for Yahoo!, Barnes & Noble and ZDNet for some time.

In 1998, Microsoft saw their opportunity. They bought Firefly and more or less stripped it for parts, transforming it into a user registration service rather than a personalization platform. It eventually became the first version of Microsoft Passport, which powers Microsoft accounts across their services and devices to this day.

Years later, Pattie Maes tried to recapture the recommendation idea with other ventures, like Media Unbound. Others did as well. None were quite able to capture the ambition of Firefly. Recommendations became largely bundled and controlled by individual websites, not tied together in a standalone application.

Some people probably remember Firefly as a proto Web 2.0 application, maybe even a bit ahead of its time. Steven Johnson, creator of early web zine FEED, once said that most Web 2.0 applications were “descendants of Firefly in one way or another”, suggesting that it’s approach and consumer eventually became a roadmap for mid-2000 Web 2.0 sites.

Someone — and come to think of it, it’s probably me — should go back and track all the core ingredients of today’s Web that were visible at Firefly circa 1996

That time has, possibly, come and gone. Nevertheless, the core ingredients of Firefly very much do live on. If you look closely at the rise and fall of Firefly you can see a faint reflection of today’s AI. The hypothetical promise, the genuine magic underlying the technology, the power of large scale datasets and collaborative filtering, the privacy concerns, the hand waving, the hype cycle, and so on.

But the web is bigger now. Much bigger. And the stakes are higher now, too. Much higher. It’s possible that many AI companies today will face the same fate as Firefly, doomed to eventual consolidation and a takeover by the major players. To never fully live up to its hype, become something that retreats into the background.

Or maybe things go differently this time. Maybe if Firefly were launched today, and not in 1996, it would succeed this time, lifted by the wings of AI.