Search results for: “yahoo”

  • Pipes by Yahoo

    I discovered something at the end of last year. The belatedly missed Yahoo Pipes was, in fact, officially called “Pipes by Yahoo.” I made that mistake, despite being well-versed in the brand guidelines, having spent a year working there with a copy consistently at my side.

    Now, why this journey down the memory superhighway? That’s a valid question. The inspiration for this post came from Bradley Horowitz’s initial post on Threads. (I had to go back and re-edit the reference to post from tweet to include it in the previous sentence, force of habit). In his post, Bradley shared the history of Pipes by Yahoo. I’m acquainted with Bradley from my time at Yahoo!. During that period, he was one of the senior executives in Jeff Weiner’s Yahoo! Search and Marketplace team.

    Consider this article as complementary to the Pipes by Yahoo history that Bradley pointed out. I will share the link where it makes sense to go over and read it in my depth. My commentary provides context prior to Pipes by Yahoo launching, the impact it had and why it’s pertinent now.

    Origins

    To comprehend Pipes by Yahoo, a fair amount of scene-setting is necessary. The contemporary web experience is now a world apart from the open web of Pipes, just as Pipes was distant from the pre-web days of the early 1990s.

    Boom to bust

    During the mid-1990s through the dot-com bust, Yahoo! generated substantial revenue from various sources, with online display advertising being the most pivotal. Launching a blockbuster film from the late 1990s to the early 2010s often involved a page takeover on Yahoo! and featuring the trailer on the Yahoo! Movies channel and Apple’s QuickTime.com. A similar approach applied to major FMCG marketing campaigns, with large display advertising initiatives.

    San Francisco billboard drive-by

    Yahoo! profited significantly during this period, as the internet was the new trend, and display advertising was a cornerstone for brand building. Money was spent generously, akin to contemporary budgets for influencer marketing programmes.

    Yahoo! occupied a space between TV, magazine advertising, and newspaper advertising. The design of the My Yahoo! page mirrored the multi-column layout of a traditional newspaper.

    Similar to a newspaper, Yahoo! developed various departments and services:

    • Search
    • News (including finance)
    • Music services
    • Shopping, featuring a store for small businesses, auctions, and a shopping mall-type offering
    • Sports
    • Communications (email, instant messaging, voice calls, early video calling)
    • Web hosting

    Then came the dot-com crash. Advertising revenue plummeted by around a third to 40 percent, depending on who you ask. Deals like the acquisition of Broadcast.com shifted from appearing speculative and experimental to extravagant wastes of money as the bust unfolded. This experience left scars on the organization, restraining the size of deals and the scope of ambition. Opportunities were second and third-guessed.

    Yahoo! Europe narrowly survived, thanks to a white-label dating product. Love proved to be a more dependable revenue source than display advertising. A new CEO from the media industry was appointed to address shareholder and advertiser concerns.

    The advertising industry was in a constant state of learning. Performance marketing emerged as a significant trend, and search advertising gained prominence.

    The initial cast in this story

    Jeff explains something to the phone

    Weiner was hired into Yahoo! by then CEO Terry Semel. Semel knew Weiner from his work getting Warner Brothers into the online space.

    Bradley

    Yahoo! had started getting serious about search by acquiring a number of search technology companies and hiring talented people in the field. Bradley Horowitz had found an image and video search startup called Virage and joined Yahoo! (a year before I got there) as director of media search.

    Tim Mayer Yahoo

    There was former Overture executive Tim Mayer who was VP of search products and drove an initiative to blow out Yahoo!’s search index as part of a feature and quality battle with Google, Bing and Ask Jeeves. It was a great product, but with the best effort in the world we didn’t have the heat. The majority of Yahoos internally used Google because of muscle memory.

    how many points for visiting the metro?

    Vish Makhijani was ex-Inktomi and was VP – international search and has more of a focus on operations. He worked on getting non-US Yahoo! users feature parity – at least in search products.

    Former Netscaper, Eckhart Walther was the VP in charge of product management.

    Aside: where did Ged sit?

    Where did I sit? Low on the totem pole. To understand my position in the organisation, imagine a Venn diagram with two interlocking circles: the European central marketing team and Vish’s team. I would have sat in the interlocking bit. If that all sounds confusing, yes it was.

    Downtown San Jose

    Search wars and web 2.0

    Pipes by Yahoo emerged from the confluence of two technological trends that developed in parallel, extending all the way to early social media platforms.

    Search wars

    I had been discussing the prospect of working at Yahoo! with a couple of people since around 2003. I had an online and technology brand and product marketing background. I had been blogging regularly since late 2002 / early 2003 and managed to incorporate online reviews and forum seeding into campaigns for the likes of Aljazeera and BT. The business was emerging from survival mode. As an outsider, it wasn’t immediately apparent how precarious Yahoo!’s situation had been. However, the threat posed by Google was undeniable.

    At that time, Google didn’t have the extensive workforce it boasts today. One of my friends served as their PR person for Europe. Nevertheless, Google had embedded itself into the zeitgeist, seemingly launching a new product or feature every week. If there wasn’t a new product, stories would sometimes ‘write themselves,’ such as the time the face of Jesus was supposedly found on Google Maps photography of Peruvian sand dunes. The closest contemporary comparison might be the cultural impact of TikTok.

    The geographical impact of Google’s cultural dominance was uneven. In the US, Yahoo! was a beloved brand that many netizens were accustomed to using. Yahoo! held double the market share in search there compared to Europe. Part of this discrepancy was due to Europeans coming online a bit later and immediately discovering Google. But Google didn’t do that well with non-Roman derived European languages like Czech. It has similar problems with symbolic languages like Korean, Chinese and Japanese.

    Google explosion

    I can vividly remember the first time I used Google. At that time I was using a hodge podge of search engines, usually starting with AltaVista and then trying others if I didn’t get what I wanted. This was before tabbed browsers were a thing, so you can imagine how involved the process became.

    Google appeared in an online article, which I think was on Hotwired some time during late 1998, less than a year after it had been founded. I clicked on a link to use the search engine. Google looked every different to now. It had a clean page with three boxes beneath. The first one was a few special searches, I think one of them was Linux-related, which tells you a lot about the audience at the time. The second was set of corporate links including a link explaining why you would want to use Google – although experiencing one search was enough for most people that I knew. The final box was to sign up to a monthly newsletter that would give updates on what developments Google was up to.

    From then on, I very rarely searched on Alta Vista, though my home page was still My Excite for a long time. This was more because I had my clients news set up on the page already and they had decent finance overage at the time.

    The difference in searches was really profound, there were a number of factors at work:

    • Google’s approach seemed to give consistently better results than the vectored approach taken by Excite or AltaVista.
    • There was no advertising on the SERP (search engine results page), but that was to soon change.
    • You could use very directed Boolean search strings, which isn’t possible any more since Google optimised for mobile.
    • Search engine optimisation wasn’t a thing yet.
    • The web while seeming vast at the time, was actually small compared to its size now. Web culture at the time was quirky and in aggregate nicer and more useful than it is now. Part of this was was down to the fact that early web had a good deal of 1960s counterculture about it. Wired magazine would write about the latest tech thing and also profile psychedelic experimenters like Alexander Shulgin. Cyberpunk, rave and psychedelic tribes blended and found a place online. You can see the carcass of this today with Silicon Valley’s continued love of Burning Man. (Note: there were rich dark seams if that was the kind of thing you were into. There wasn’t the same degree of social agglomeration that we now have, nor were there algorithms that needed constant new content to feed diverse realities.)
    • Content creation on the web was harder than it is now. Blogging was at best a marginal interest, the likes of Angelfire, AOL Hometown, Geocities and Tripod provided free hosting, but you couldn’t put up that much content to pollute the search index even if you wanted to.

    The impact was instantaneous and by early 1999, it was much a part of the nascent netizen culture as Terence McKenna.

    Homage to Terence McKenna

    McKenna spent the last bit of his life interrogating the search engine for four to five hours a day. He was convinced that the online world it provided access to represented some sort of global mind.

    Sometimes he treats the Net like a crystal ball, entering strange phrases into Google’s search field just to see what comes up. “Without sounding too cliché, the Internet really is the birth of some kind of global mind,” says McKenna. “That’s what a god is. Somebody who knows more than you do about whatever you’re dealing with.”

    As our society weaves itself ever more deeply into this colossal thinking machine, McKenna worries that we’ll lose our grasp on the tiller. That’s where psychedelics come in. “I don’t think human beings can keep up with what they’ve set loose unless they augment themselves, chemically, mechanically, or otherwise,” he says. “You can think of psychedelics as enzymes or catalysts for the production of mental structure – without them you can’t understand what you are putting in place. Who would want to do machine architecture or write software without taking psychedelics at some point in the design process?”

    Terence McKenna’s Last Trip – Wired.com (May 1, 1999)

    A year after that McKenna interview, Google was running over 5,000 Linux servers to power the search engine.

    At first, Google also powered search on some of the web portals and saw itself as a competitor to search appliance businesses like Inktomi and Autonomy. The advertising kaiju started operation in 2000 and it was tiny. This violated patents held by GoTo.com – a business subsequently acquired by Yahoo!.

    Post-bust

    Once Yahoo! had disentangled itself from the carnage of the dot com bust, search was a much bigger deal. And Google had become a behemoth in the space of a few years. In 2002, Google launched Google News – a direct challenge to web portals like Yahoo!, MSN and Excite. Around about this time Google started to be used as a verb for using a web search engine.

    While display advertising had taken a dive, search advertising had took off for several reasons:

    • It was performance marketing, even when a business is just surviving sales are important
    • Behavioural intent – if you were searching for something you were likely interested in it and may even purchase it
    • So easy to do at a basic level, even small and medium sized businesses could do it
    • Advertising dashboard – Google did a good job at helping marketers show where the advertising spend had gone.

    We’ll ignore on the difficult facts for the time being, for instance:

    • The role of brand building versus brand activating media
    • What attribution might actually look like
    • That Google advertising is a rentier tax, rather than a business generator

    Google listed on the stock market in August 2004. Investors ignored governance red flags like the dual share structure so the founders could retain voting rights.

    Yahoo! in the search wars

    Yahoo! had come out of the dot com bust battered but largely intact. Yahoo! was scarred in a few important ways.

    Identity crisis

    Yahoo! came about pre-Judge Jackson trial when Microsoft spread terror and fear into the boardroom of most sensible technology companies. I know that sounds weird in our iPhone and Android world. Rather than the bright cuddly people who give us Xbox, it was a rabid rentier with a penchant for tactics that organised crime bosses would have approved of. It took a long time to work that out of their system.

    Another big factor was the fear of Microsoft. If anyone at Yahoo considered the idea that they should be a technology company, the next thought would have been that Microsoft would crush them.

    It’s hard for anyone much younger than me to understand the fear Microsoft still inspired in 1995. Imagine a company with several times the power Google has now, but way meaner. It was perfectly reasonable to be afraid of them. Yahoo watched them crush the first hot Internet company, Netscape. It was reasonable to worry that if they tried to be the next Netscape, they’d suffer the same fate. How were they to know that Netscape would turn out to be Microsoft’s last victim?

    Paul Taylor – ex Yahoo and founder of Y-Combinator

    Yet Yahoo! went on to hire media mogul Terry Semel as it went through the dot com bust, shows that this thinking must have coloured views somewhat.

    Cheque book shy

    Even Mark Cuban would admit that Broadcast.com was not worth the billion dollar price tag that Yahoo! paid for it. It was a high profile mistake at the wrong point in the economic cycle which haunted Yahoo! acquisition plans for years. Which is one of the reasons why may have Yahoo! dropped the ball when it had the chance to buy Google and Facebook.

    The game has changed

    But the game had changed. Display advertising was no longer as profitable as it had been. Search advertising was the new hotness, fuelled by online commerce. By early 2004, Yahoo! is confident enough in its own search offering to drop Google who had been providing its search function.

    Yahoo! acquired search appliance business Inktomi in 2002 and then Overture Services in 2003. Overture services provides the basic ad buying experience for Yahoo! search advertising.

    In 2004, Yahoo! realises having search is not enough, you have to offer at least as good as product as Google, if not better. This is where Tim Mayer comes in and for the next couple of years he leads a project to build and maintain search parity with Google.

    You had a corresponding project on the search advertising side to bring the Overture buying experience up to par with Google with a large team of engineers. That became a veritable saga in its own right and the project name ‘Panama‘ became widely known in the online advertising industry before the service launched.

    Search differently

    Googling is a habit. In order to illicit behavioural change you would have to

    • Have an alternative
    • Change what it means to search in a positive way

    Yahoo! approached this from two directions:

    • Allowing different kinds of information to be searched, notably tacit knowledge. I worked on the global launch of what was to become Yahoo! Answers, that was in turn influenced by Asian services notably Naver Knowledge IN. This approach was championed internally by Jerry Yang.
    • Getting better contextual data to improve search quality providing a more semantic web. This would be done by labels or tags. In bookmarking services they allowed for a folksonomy to be created. In photographs it provided information about what the pictures or video content might be, style or genres, age, location or who might be in them.

    Web 2.0

    Alongside a search war there was a dramatic change happening in the underpinnings of the web and how it was created. While the dot com bust caused turmoil, it also let loose a stream of creativity:

    • Office space was reasonably priced in San Francisco only a couple of years after startups and interactive agencies had refurbished former industrial buildings South of Market Street (SoMo).
    • Office furniture was cheap, there was a surplus of Herman Miller Aeron chairs and assorted desks floating around due to bankruptcies and lay-offs.
    • IT and networking equipment was available at very reasonable prices on the second hand market for similar reasons. You could buy top of the range Cisco Catalyst routers and Sun Microsystems servers for pennies on the dollar that their former owners had paid for them less than one computing generation before. This surplus of supplies be bought online from eBay or GoIndustry.com.
    • Just in time for the internet boom wi-fi had started to be adopted in computers. The first wi-fi enabled laptop was the Apple iBook. Soon it became ubiquitous. Co-working spaces and coffee shops started to provide wi-fi access connected to nascent mainstream broadband. Which meant that your neighbourhood coffee shop could be a workspace, a meeting space and a place to collaborate. We take this for granted now, but it was only really in the past 25 years that it became a thing. It also didn’t do Apple’s laptop sales any harm either.
    • Open source software and standards gave developers the building blocks to build something online at relatively little financial cost. Newspapers like the Financial Times would have spent 100,000s of pounds on software licences to launch the paper online. In 2003, WordPress was released as open source software.
    • Amazon launched its web services platform that allowed developers a more flexible way for putting a product online.
    • The corresponding telecoms bust provided access to cheaper bandwidth and data centre capacity.

    All of these factors also changed the way people wrote services. They used web APIs building new things, rather than digital versions of offline media. APIs were made increasingly accessible for a few reasons:

    • Adoption of services was increased if useful stuff was built on top of them. Flickr and Twitter were just two services that benefited from third party applications, integrations and mashups. Mashups were two or more services put together to make something larger than the ingredients. The integration process would be much faster than building something from scratch. It worked well when you wanted to visualise or aggregate inputs together.
    • Having a core API set allowed a service to quickly build out new things based on common plumbing. Flickr’s APIs were as much for internal development as external development. Another example was the Yahoo! UK’s local search product combining business directory data, location data and mapping.
    local

    There was also a mindset shift, you had more real-world conferences facilitating the rapid exchange of ideas, alongside an explosion of technical book publishing. One of the most important nodes in this shift was Tim O’Reilly and business O’Reilly Publishing. Given O’Reilly’s ringside seat to what was happening, he got to name this all web 2.0.

    Finally, a lot of the people driving web 2.0 from a technological point of view were seasoned netizens who had been exposed to early web values. The following cohort of founders like Mark Zuckerberg were more yuppie-like in their cultural outlook, as were many of the suits in the online business like Steve Case or Terry Semel. But the suits weren’t jacked into the innovation stream in the way that Zuckerberg and his peers – but that would come later.

    This was the zeitgeist that begat Pipes by Yahoo.

    The approach to a new type of search needed the foundational skills of web 2.0 and its ‘web of data’ approach. Yahoo! acquired number of companies including Flickr, Upcoming.org and Delicious. At the time developers and engineers were looking to join Yahoo! because they liked what they saw at Flickr, even though the photo service was only a small part of the roles at the business.

    Web 2.0 talent

    The kind of people who were building new services over APIs were usually more comfortable in a scrappy start-up than the large corporate enterprise that Yahoo! had become. Yet these were the same people that Yahoo! needed to hire to develop new products across knowledge search, social and new services.

    There were some exceptions to this, for instance the 26-person team at Whereonearth who operated a global geocoded database and related technology had a number of clients in the insurance sector and Hutchison Telecom prior to being acquired by Yahoo!. The reason why Yahoo! became so interested was a specific Whereonearth product called Location Probability Query Analyser. The technology went on to help both the Panama advertising project and Yahoo! search efforts. George Hadjigeorgiou was tasked with helping them get on board.

    I knew some of the first Flickr staff based out of London, they sat alongside technologist Tom Coates who would later work on FireEagle. They all sat in a windowless meeting room on a floor below the European marketing team sat in.

    Most people didn’t even know that they were there, working away thinking about thinks like geotagging – a key consideration in where 2.0 services and mobile search.

    Going over to the Yahoo! campus in Sunnyvale made it clear to me that the difference in cultural styles was equally different over there, from just one cigarette break with Stewart Butterfield of Flickr.

    Secondly, there was the locale. The best way I found to help British and Irish people get the environment of Silicon Valley was to describe it as a more expansive version of Milton Keynes with wider roads and a lot more sunshine. One of the biggest shocks for me on my first visit to the Bay Area was how ordinary Apple and Google’s offices felt. (This was 1 Infinite Loop before Apple Park construction started). The canopy over the main building entrance looked like an airport Novotel, or every shopping centre throughout the UK.

    In the same way that Milton Keynes is not London; Silicon Valley’s quintessential campus laden town Sunnyvale is not San Francisco.

    This is not the dystopian doom spiral San Francisco city of today with failed governance and pedestrianisation projects. At this time, San Francisco was on the up, having been clobbered by the dot com bust in the early noughties, financial services had kept the city ticking over. Technology was on the rise again. Home town streetwear brand HUF was making a name for itself with its first shop in the Tenderloin, the DNA Lounge had consistently great nights from west coast rave and goth sounds to being a haven for mashup culture with its Bootie nights.

    There was great cinemas, vibrant gay night life and the sleaze of the Mitchell Brothers O’Farrell theatre. The Barry Bonds era San Francisco Giants won more than their fair share of baseball matches.

    If Yahoo! were going to keep talent, they’d need a place in the city. It makes sense that setting up the San Francisco space fell to Caterina Fake. Fake was co-founder of Flickr and was given a mandate by Jerry Yang to ‘make Yahoo! more like Flickr’. So she decided to set up an accelerator for new products.

    Brickhouse

    According to Caterina Fake on Threads:

    I dug around on the company intranet and exhumed an old deck for an initiative called “Brickhouse” which had been approved by the mgmt, but never launched.

    Caterina Fake (@cefake on threads)

    This tracks with my experience in the firm, projects would form make rapid progress and then disappear. And during the first dot com boom, San Francisco was home to online media companies, such as Plastic (Razorfish SF), Organic and Agency.com, many of whom also had offices in New York. Wired magazine had its office there, as did a plethora of start-ups.

    Fake goes on to say that Brickhouse managed to use the same office space she had worked in while she had worked at Organic over a decade earlier.

    The 60 Minutes episode Dot-com Kids marked an acme in this evolution of San Francisco. At the time Fake was doing this exercise, there was probably a Yahoo! sales team based in San Francisco proper, but that would be it.

    Fake cleans up the Brickhouse deck and gets it through the board again with Bradley Horowitz with the then Chief Product Officers Ash Patel and Geoff Ralston, president Sue Decker and chief Yahoo Jerry Yang being the board champions of the project.

    Fake hands off to Chad Dickerson to realise Brickhouse as she heads off on maternity leave. Fake, Dickerson and Horowitz assemble the Brickhouse team (aka the TechDev group) and ideas that would eventually build Pipes by Yahoo!, Fire Eagle and other projects.

    This is where my origins viewpoint on Pipes by Yahoo finishes. For the download on its creation, go here now; the link should open in a new tab and I will still be here when you get back to discuss the service’s impact.

    Pipes by Yahoo was launched to the public as a beta product on February 7 2007. Below is how it was introduced on the first post added to the (now defunct) Yahoo Pipes Blog. At this time product blogs became more important than press releases for product launches as information sources to both tech media and early adopters.

    Introducing Pipes

    What Is Pipes?
    Pipes is a hosted service that lets you remix feeds and create new data mashups in a visual programming environment. The name of the service pays tribute to Unix pipes, which let programmers do astonishingly clever things by making it easy to chain simple utilities together on the command line.

    Philosophy Behind the Project
    There is a rapidly-growing body of well-structured data available online in the form of XML feeds. These feeds range from simple lists of blog entries and news stories to more structured, machine-generated data sources like the Yahoo! Maps Traffic RSS feed. Because of the dearth of tools for manipulating these data sources in meaningful ways, their use has so far largely been limited to feed readers.

    What Can Pipes Do Today?
    Pipes’ initial set of modules lets you assemble personalized information sources out of existing Web services and data feeds. Pipes outputs standard RSS 2.0, so you can subscribe to and read your pipes in your favorite aggregator. You can also create pipes that accept user input and run them on our servers as a kind of miniature Web application.

    Here are a few example Pipes to give you an idea of what’s possible:

    • Pasha’s Apartment Search pipe combines Craigslist listings with data from Yahoo! Local to display apartments available for rent near any business.
    • Daniel’s News Aggregator pipe combines feeds from Bloglines, Findory, Google News, Microsoft Live News, Technorati, and Yahoo! News, letting you subscribe to persistent searches on any topic across all of these data sources.

    What’s Coming Soon?
    Today’s initial release includes a basic set of modules for retrieving and manipulating RSS and Atom feeds. With your help, we hope to identify and add support for many other kinds of data formats, Web services, processing modules and output renderings.

    Here are some of the things we’re already got planned for future releases:

    • Programmatic access to the Pipes engine
    • Support for additional data sources (such as KML)
    • More built-in processing modules
    • The ability to extend Pipes with external, user-contributed modules
    • More ways to render output (Badges, Maps, etc…)

    Pipes is a work in progress and we’ll need your help to make it a success. Try building some simple pipes and advise us what works well and what doesn’t in the online editor. Tell us how you’d like use Pipes, what we can do to make cool things possible, and show us ways you’ve found to use Pipes that never even occurred to us. In return, we promise to do our best to make Pipes a useful and enjoyable platform for creating the next generation of great Web projects.

    And please have fun!

    The Pipes Development Team

    Pipes impact

    I had a good, if exhausting time at Yahoo! It was first inhouse role and my part of the central marketing team had an exhausting workload. By the time Pipes by Yahoo launched, I had left Yahoo! Europe. There has been a re-organisation of European arm and the business had been ‘Kelkoo-ised’; a few of us on the European central marketing team took the opportunity to take the money and run.

    I remember bringing Salim (who headed the European search team) up to speed and getting his support to push for me getting a payout, rather than fighting my corner.

    Peanut Butter Memo

    Brad Garlinghouse’s peanut butter manifesto was made public towards the end of the year portraying a game of thrones type power play which would have seen the kind of structures that were put in place in the European organisation rolled out globally.

    On the face of it, some of it was pertinent, but it lacked a wider vision.

    While Garlinghouse has gone on to have a really successful career at Ripple; the Yahoo! business unit he ran had several problems. He was in charge of Music and the Comms & Community BU. At the time it had a poor record of building products fit for early adopters like music properties that aren’t Mac-compatiable, this was when the iTunes store and Apple iPod springboard off the Mac community and into the mainstream.

    The then new Yahoo! Mail which didn’t work on Safari and a Messenger client which was worse to use than third party clients like Trillium or Adium. All of which made it hard to build a buzz that will bridge to mainstream users. Yahoo! Messenger, could have been Skype or WhatsApp. It became neither.

    For a more modern example, think about the way Instagram and Threads were Apple iPhone first to build a core audience.

    At the time, I was less charitable about the memo. And the memo raised wider questions about the business; like was the CEO facing an executive revolt?

    The launch of Pipes by Yahoo helped to inject some more positive energy back into the Yahoo! brand. Remember what I said earlier on how talent wanted to join Yahoo!’s engineering and development teams because of Flickr. They started to want to join Yahoo! because of Pipes.

    The outside world

    I was back agency side when Pipes launched. I had friends within Yahoo! still and kept an eye on the various product blogs. I got the heads-up on Pipes and put aside an afternoon and an evening to explore it fully. A quick exploration gave one an idea of how powerful Pipes by Yahoo could be. While Pipes was powerful, it was also relatively user friendly, like Lego for data. It was more user friendly than Apple’s Automator, which inspired Pipes by Yahoo! in the first place.

    At this time in London the amount of people working on social media and online things was still relatively small. Knowledge was shared rather than hoarded at grassroots events and on an ecosystem of personal blogs. This was a group of eople with enquiring minds, a number of whom I can still call friends.

    We shared some of the public recipes on Pipes by Yahoo and learned from them, just as I had learned about Lotus 1-2-3 macros in the early 1990s, by picking through other peoples examples. (I put this to use automating data records in the Corning optical fibre sales support laboratory that I worked in at the time.)

    The agency I worked with had a number of large technology clients including AMD, Fujitsu Siemens personal computing devices – notably smartphones, parts of Microsoft and LG.

    AMD and Microsoft were keen to keep track on any mention of their brand in a number of priority blogs or news sites at the time. Social listening was in its infancy and there were a number of free tools available, which I got adept at using.

    We managed to build and sell both AMD and Microsoft respectively a custom feed which provided them with links to relevant content in near real-time, which they then published on an internal site so that key audiences always had their fingers on the pulse.

    This was all built on top of two free Pipes by Yahoo accounts which used a similar but tweaked recipes to make this happen.

    On the back of that work, we managed to sell in a couple of small websites to the Microsoft team based on WordPress. I had long moved on to another agency role by the time the Pipes by Yahoo feeds would have died.

    Discussing Pipes by Yahoo with friends, they said it had inspired them to learn to code. Pipes by Yahoo spurred creativity and creation in a similar way to HyperCard.

    Zeitgeist

    While all of this has talked about Pipes by Yahoo! and how great the launch was, the ending of Pipes was much more humdrum. The service had been glitchy at the best of times and wasn’t being maintained in the end. In conversations I had with friends, it was compared to a British sports car: unreliable but loveable. Yahoo! closed it down on September 30, 2015.

    Which begs the question, why is Pipes by Yahoo, which was shut down eight and a half years ago being celebrated amongst the digerati?

    I think that the answer to this is in the current online zeitgeist. The modern web isn’t something that anyone involved in web 2.0 would have signed up for. Algorithms have fragmented the global town hall archetype envisaged for social. The web no longer makes sense in aggregate, as it’s splintered by design.

    The modern web feels ephemeral in nature. This seems to have gone hand-in-hand with a video first web exemplified by TikTok.

    The social platforms the fragmentation seem to be declining in relevance and its isn’t clear what’s next. The people-driven web of knowledge search and web 2.0 is under pressure from AI content providing a mass of ‘just good enough’ content. Even influencers are being usurped by digital avatars. Even the audience engagement is often synthetic. All of which leaves the netizen in a state of confusion rather than the control that Pipes by Yahoo offered.

    Taylor Lorenz is a journalist who made net culture and platforms her beat. Taylor Lorenz’ book Extremely Online feels like she is reporting from another planet rather than the recent web and it was published in October last year.

    More information

    Mediasaurus no more? The Well

    Let’s Get This Straight: Yes, there is a better search engine | Salon.com (December 21, 1998)

    The Original GOOGLE Computer Storage Page and Brin

    Notre histoire en détail | Google

    How Google Became a Verb | TLF Translation

    Facebook Yahoo! patents case | renaissance chambara

    Yahoo! Answers Adoption | renaissance chambara

    Sadowski, J. (2020). “The Internet of Landlords: Digital Platforms and New Mechanisms of Rentier Capitalism.” Antipode 52 (2): 562-580.

    Amazon.com Launches Web Services; Developers Can Now Incorporate Amazon.com Content and Features into Their Own Web Sites; Extends ”Welcome Mat” for Developers | Amazon.com newsroom

    Nobody Knows What’s Happening Online Anymore – The Atlantic

    Extremely Online: The Untold Story of Fame, Influence and Power on the Internet by Taylor Lorenz

    The Age of Social Media Is Ending | The Atlantic

    AI is killing the old web, and the new web struggles to be born | The Verge

    Is the web actually evaporating? | Garbage Day

  • Yahoo Answers

    Back in 2005, I worked in the search group at Yahoo!. One of the projects that I worked on was Yahoo Answers. 16 years later, Yahoo Answers is being closed down. I thought I would capture some of my memories and inside knowledge on Yahoo Answers.

    But first we need some context so that what I write later about Yahoo Answers will make sense.

    The beginning

    Let’s go back to the beginning. Back to the early-1990s. Jerry Yang and David Filo founded Yahoo!. It fits the classic Silicon Valley archetype story and you can find plenty of accounts of it elsewhere. The key is what Yahoo! originally was. Its a list of links for websites. Once the list grew above 200 links or so; Jerry and David came up with a way of displaying this list by grouping it into subject areas.

    Jerry & David, 1995

    What would later be called a web directory. There were other directories around about this time like:

    • Best of the Web – which surprisingly still exists
    • Netscape Communications had their own directory when they acquired Gnuhoo, this eventually became DMOZ and then Curlie. Gnuhoo did rely on a search engine to help you find things in their directory. This is available as open source code at GitHub

    All of them had a certain amount of editorial input over what was good. Yet Yahoo! became the top one through buzz marketing – cheap ways to do brand building.

    When I was there, I worked with an agency to organise event hijacking at the Harry Potter and the Half Blood Prince book launch at Waterstones flagship store on Oxford Street. Yahoo! would vinyl wrap any employee’s car for free. There were also strategically placed billboards, such as this one in San Francisco.

    San Francisco billboard drive-by

    People who managed this directory were known as web surfers. But there was also search engines out there, like the Knowbot search engine for Telnet developed in the late 1980s. There was Archie which was the closest to what we think of as a search engine now. Archie searched FTP archives around the world.

    As computer science post-grad students, Filo and Yang would have been familiar with the idea of the search engine. At the time David Filo felt that no machine would provide better filtering than a human. Media accounts of the time showed that Silicon Valley venture capitalists were all in favour of search engines over directories.

    Peer companies like:

    • Webcrawler
    • Metacrawler
    • Lycos
    • Ask Jeeves
    • Infoseek
    • Excite
    • AltaVista

    All offered what we’d recognise as reasonable search experiences. But Filo’s comments on human filtering is something that we will revisit later.

    Web portal & web advertising

    Search engines were the future but as the dot com era took off it wasn’t apparent how to monetise them.

    Yahoo! early morning of March 3, 1999
    Yahoo! home page early on the morning of March 3, 1999

    At the height of the dot com era; Yahoo! had about 40 million users a month. You have to remember there weren’t that many people online in comparison to now. Internet usage had grown from 45 million users in 1995 to over 410 million by 2000. At the time it didn’t seem to matter that Yahoo! took longer to load as a website compared to its peers. Longer page times, meant that you could get away with less equipment in your data centre hosting the website and supporting infrastructure.

    The internet didn’t give birth to culture in the same way that memes, influencers and platforms do now. Instead it was the meme. It was all over the mainstream media, often tied up with ideas of cyberpunk culture, bulletin boards and the ‘information superhighway’. Examples of this included:

    • The Site by MSNBC
    • The i in iMac was for internet. The idea was that you could take the computer out of its box, plug it in to your wall socket and telephone socket. When you turned it on, it would configure you an internet service. The cool product design was a byproduct of this internet appliance plus personal computer thinking
    • Movies: The Lawnmower Man, Hackers, The Matrix, Ghost In The Shell
    • Books: Snow Crash, William Gibson’s Neuromancer
    • A plethora of internet magazines, including Ziff-Davis’ Yahoo! Internet Life which was a mix of technology and gadget reviews, media and celebrity content and website recommendations. Yahoo! Internet Life was published from 1996 to 2002

    It felt like something big was going to happen, even if we didn’t know what it was. What was obvious was the potential for advertising online. And the clearest analogue was newspaper advertising due to the long page format of web pages.

    Web portals came about for a number of reasons:

    There was now the technology to pull content from different sources together. You would have:

    • Weather forecast
    • Horoscope
    • Up to date news
    • Local information (for major cities like San Francisco)
    • Business
    • Finance
    • Entertainment and celebrity news

    Like the newspaper before it, it offered the first media you needed, but on the web.

    It was mainstream enough for brands to advertise against for brand building.

    By the time I was leaving college, Yahoo! Mail accessed through the Yahoo! home page made perfect sense.

    So before the dot com bubble bursts Yahoo! had a major media business valued at 2.8 billion dollars, or about $70 dollars per user. Which sounds expensive, but when you consider that Google is now worth about $386 per user, it’s not that bad. Secondly, online advertising per impression was much more lucrative back then and ad fraud was much less of an issue.

    What there wasn’t was a way of taking advantage of the highly relevant search results provided by search engines and adequately monetising them. So companies had three ways of monetising search:

    • Companies created portals the so called ‘homepages of the web’ to put display adverts on like My Yahoo! or MSN.com and search was a service alongside news, weather and horoscopes
    • They became infrastructure companies selling search functionality in the background a la Inktomi
    • They sold inclusion in their directory. This was controversial as it went against the editorial integrity of the directory and still a hot button when I arrived at Yahoo! in 2005

    The bubble bursts

    In the US stock market we had was now known as the internet or dot.com bubble. Looking at the NASDAQ composite data, it seemed to start in the last quarter of 1995, six months or so before Yahoo! went public in April 1996. It reached its nadir in the last quarter of 2002.

    In reality, this was more than about websites. Telecoms deregulation, satellite networks and the rise of cellphones had seen a boom in new companies and network equipment providers to support them. The need for servers had created booms in:

    • Computers: SGI, Sun Microsystems and IBM
    • Networking equipment: US Robotics, 3Com, Cisco
    • Software: VA Linux, RedHat, Open Text
    • Software as a service: I2, Salesforce, NetSuite
    • Web hosting and ‘data hotels’: Equinix, Intel, Rackspace, PSINet
    • Telecoms and ISPs: Level3, Global Crossing, Earthlink, Iridium, GlobalStar, AOL, @Home Network
    2560px-Nasdaq_Composite_dot-com_bubble.svg
    NASDAQ composite index covering the dot com boom and crash

    Add into that artificially high growth in earnings for enterprise IT companies in the run up to the Y2K bug issue and the whole sector was left with a bad hangover.

    Eric Steiner tells his tale as the CEO of Inktomi in 2004

    Steiner’s talk is interesting because it shows how the search business, selling search capability to the likes of Microsoft, Amazon and eBay had slow and steady growth rather than outstanding growth during this time.

    Yahoo! went through a traumatic time. When I worked at Yahoo! Europe, I was told online advertising sales dropped to a third of what they were during the dot com boom. The European business managed to hold on by its finger tips thanks to revenues from online dating services.

    Some of the ‘smart bets’ Yahoo! made during the boom times looked like hubris. The exemplar of this was Yahoo!’s acquisition of Broadcast.com. Broadcast.com provided video streaming (then called web casting) and internet radio services. It was the technology partner for the first online Victoria Secret Fashion Show streamed online. Yahoo! acquired it for 5.6 billion of Yahoo! stock. This was a bad decision, but thankfully, they didn’t pay cash.

    When I joined Yahoo! the Broadcast.com acquisition was still a scar on acquisition decision-making. You can attribute the impact of this to subsequent failed purchases of Google and Facebook.

    GoTo and Google

    In 1998, the company GoTo.com launched paid advertising placement in search engine results. The next year they introduced real time bidding. It was renamed Overture and started providing these services for Yahoo! and others. It started to become successful as a business.

    Meanwhile, Google had moved from a research project to a serious search engine. In 2000, Google began selling advertisements associated with search keywords. This was against Page and Brin’s initial opposition toward an advertising-funded search engine, they saw themselves more as a ‘search appliance’ business rather like Inktomi. Yahoo! adopted Google search around about the same time that Google started its search advertising business.

    This put Google in front of a large number of consumers and helped Google further refine its search engine.

    Google’s own offering was the exact opposite of Yahoo!. It prided itself its clean design with just a search box. Google also had a fanatical obsession with reducing page load times and the time taken to return search results.

    This was what more and more people wanted. Google used the dot com crash to build its business and its infrastructure. It wasn’t until its 2004 IPO that rivals realised how much of a head start Google had.

    Google revolutionised data centre server design, reducing cost and increasing the amount of servers that it could use. By contrast every Yahoo! data centre hardware purchase went via David Filo. If you used Yahoo! small business hosting, you were using tired and almost expired Yahoo! servers. In retrospect, they looked after the datacentre pennies, but let the pounds slip away.

    2003 saw Yahoo! get serious about the search engine business. The company purchased Overture which included GoTo.com and Altavista. But the problem was that even if Yahoo! built a search engine as good as Google, it didn’t matter if people didn’t use it. During my time at Yahoo! there was a push to get the necessary servers in place and a product that was as good as Google. However there was a constant tit-for-tat feature development in the search space. By this time Google had already verbed. The Google habit means that its hard to compete against them.

    I heard that inside Microsoft they tried to take drastic measures to persuade employees to use Bing over Google. When I worked at Yahoo! people used Google a lot too.

    The only way to compete with Google was to have a different idea. Google defined its mission is to organize the world’s information and make it universally accessible and useful.

    Yahoo! needed a new idea that was distinct from Google’s mission. The idea was knowledge search.

    Knowledge search and Yahoo! Answers

    Knowledge search as a concept was well under way by the time that I arrived. It was to capture and make searchable all the ‘knowledge’ (rather than information in the world). Opinions, experience and recommendations are knowledge rather than information. Yahoo!’s web 2.0 acquisitions including Flickr and delicious were made to support this vision.

    Tagging built up words and associations with web links and images, effectively human filtering – some of which would be used to train machine learning algorithms. The next logical step would be to build a repository of knowledge by the people, for the people. That’s where Yahoo! Answers came in.

    The inspiration for Yahoo! Answers came from a product that Yahoo! Taiwan had rolled out. It in turn probably inspired by Korean site Naver Knowledge IN. Bradley Horowitz apparently claimed that Yahoo! Answers was inspired by Naver Knowledge IN directly.

    Knowledge IN was designed to encourage user created content, since there wasn’t much material on the Korean web at the time.

    When I heard Jerry Yang talk about it internally at the time, he talked bruskly about a product built by Yahoo! Taiwan as having inspired it. Jerry didn’t do jet lag well and came across as morose on the couple of times I saw him in Europe, so wasn’t exactly an effusive speaker.

    Yahoo! Answers was championed by Jerry and that blessing allowed it to be pushed through when so many other product died before they got pushed to beta. It makes sense to point out the human crafted nature of Yahoo! Answers. In this respect it can be seen as a direct line back to the original Yahoo! directory product. Both were fuelled by a belief that people had some ability that was better than machines.

    Qi Lu was responsible for new products within the core search business and the troubled Panama search advertising project at the time. Weekly conference calls saw a plethora of existing projects cancelled, or reprioritised by Qi Lu, while new ones would suddenly appear. This constant change in the roadmap mean’t a lot of wasted efforts.

    Yahoo! Answers and much of the knowledge search related acquisitions sat under Bradley Horowitz. Tim Mayer was focused on the commercial side of things, although there was some overlap in the roles. Eckhart Walter sat above Tim. Jeff Weiner was the main shot caller having both Search and Marketplace businesses reporting into him. If you’re thinking, that’s a lot of senior management involved. You’d be right, there were a lot of managers with varying degrees of responsibility involved.

    But they were all good people and I’d be happy to work with them again.

    Prior to Yahoo!,I had been working agency side for Transversal. Transversal powered the support functions for a number of companies including Sony Playstation. I had a good idea how much this service was priced and floated the idea of sponsored channels for instance around Sony Playstation and had a good idea how much Sony must be paying to support user troubleshooting.

    But it didn’t fit that well as an idea with knowledge search.

    Concerns and how is babby formed?

    In the European team we had some concerns about Yahoo Answers like how was it going to get monetised? The quality of the content was also a concern. Knowledge IN and similar services in Asia work partly due to culture. We were worried when it hit a more individual-focused culture like the US or Europe.

    Another problem was calibrating the rewards within the system. Its really hard to get the balance on good quality questions and answers. Generally people who are time rich, aren’t necessarily the best respondents. If you need one proof point to show how much of a failure this was, you only have to look at the how is babby formed? meme.

    Rewards aren’t the only problem however. The second issue was the way the community was built. Generally, a great community is built carefully from like-minded people. With flickr it was around the passion of photography. Facebook is actually closer to Reddit, built on groups of groups. The death of a group dynamic won’t necessarily kill the platform.

    I was involved in early seeding of the initial content on Yahoo! Answers. I answered 42 questions, the first one question I answered was ‘What to take from airport to downtown Munich?‘ My response: The taxi is reasonable, it cost me 30 Euros – which shows the contextual nature of knowledge search. 30 Euros was reasonable for me at the time, since I could expense it back, but it wouldn’t be reasonable for a backpacking traveller.

    I also wrote six questions, the first one was ‘Has anybody got a Pentax K100D, if so what do you think of it? What are its pitfalls and what aspects of it do you particularly like? I wanted to get a a bit more colour beyond the reviews I’ve read online. – I was getting ready to leave Yahoo! and was going to buy a DSLR camera to take better pictures on my Flickr account. I deliberately structured the question to get opinions from early users. The Pentax K100 had recently been launched.

    Careful community management is at odds with a platform trying to capture the world’s knowledge. So the Yahoo Answers community was built for rapid global user growth. For the English language versions at least, there was a global content index, sitting on top of a distributed Oracle database.

    This meant a clash of cultures and variable quality content. I quickly found the site unusable for productive questions. Yahoo! spent the next few years trying to perfect it. People that formerly worked on Yahoo! Directory and front page brought their content and editorial skills to bear on Yahoo! Answers.

    I suspect that trying to monetise the service would have been a constant challenge. Yahoo! Answers provided variable quality answers for children’s homework and was the butt of memes. Neither of which are an ideal recipe for the kind of content large brands like Procter & Gamble would want to put their name against.

    Quora’s lean pickings

    Google tried to do it better with Google Knol and also failed.

    Quora was formed in 2009 and managed to build a better community, but I’ve still seen a steady decline in the quality of their answers. In 2019, they had a user base of 300 million people and total revenue (from advertising) of 20 million dollars. Thats an ARPU of 6.6 cents. That’s not a good internet media business. From that 20 million, they need to pay their infrastructure costs, maintain and improve the product, pay the salaries of their 300 employees. And I haven’t even talked about how their investors must feel.

    Knowledge search is still a technology challenge waiting to be conquered.

    More information

    Yahoo: a history of the internet in 5 acts – Financial Times (July 25, 2016)

    Max Roser, Hannah Ritchie and Esteban Ortiz-Ospina (2015) – “Internet”. Published online at OurWorldInData.org.

    How Yahoo! Won The Search Wars Once upon a time, Yahoo! was an Internet search site with mediocre technology. Now it has a market cap of $2.8 billion. Some people say it’s the next America Online. by Randall E. Stross – Fortune (March 2, 1998)

    Britannica Online – authoritative top level history of Yahoo!

    Room for the Internet; Combining a Data Center With a ‘Telco Hotel’” – New York Times (May 14, 2000)

    Take Naver Global Today! – Korea IT Times

  • Yahoos and things this week

    Catching up with fellow former Yahoos. Employees of Yahoo! were known as Yahoos. All three of us worked on the European marketing team in London at the time when web 2.0 was kicking off. It was great to see my fellow former Yahoos. We kicked around the move away from quality in online media, historic ‘pirated television reception’ in Ireland, the worrying state of brand marketing and new ventures. If you are looking for high-quality male grooming products Charles has been working on Scrubd.

    tres Yahoo! deliquentes

    Andrew Tuck tells the Monocle story on the Mediamasters podcast. Really interesting origin story – what’s interesting is the contrarian thinking of Tyler Brûlé. Monocle is notable for building a print magazine in the face of a digital onslaught of media. More media related content here.

    Great interview with Roger Smith about the wonders of mechanical watch.

    Smith worked with Dr. George Daniels who invented the co-axial movement which has been a major step forward in mechanical watch making – changing the way power is delivered that is more efficient and can improve accuracy.

    Smith is exceptionally articulate about the technical differences of the different watches from high accuracy quartz watches to classic Rolex and Omega movements.

    Antonio Da Silva’s Hell’s Night where famous film characters interact in one night club stunned me when I saw it a few years ago. In the meantime he did a sequel and has had time to work on the colour grading of the video to make it even more mesmerising.

    Hell’s Club

    Hell’s Club Two

    Finally I am surprised that this was something I hadn’t heard already – what if the late great Terry Wogan had listened to Skibadee or MC GQ vintage mixtapes and Rinse FM? Peter Serafiniwicz channels The TWOG$.

    SaveSave

    SaveSave

    SaveSave

    SaveSave

  • Friends from Yahoo!

    The highlight of this week was briefly catching up with a couple of friends from Yahoo!. The people that I met along the way at Yahoo! were great and I am proud to say that I have more than a couple of friends from Yahoo! The company got culture right, but board level business wrong.

    I didn’t get to watch The Oscars, but even I was aware that it was FUBAR’d. The Mobile World Congress saw the launch of new handsets by Sony, LG and Huawei – but were drowned about by Nokia reinventing their iconic 3310 feature phone. This showed that smartphone manufacturers had crafted their products to a high degree and no longer came up with products that amazed us. They are all now much-of-a-muchness.

    Rolex broke out an advertising campaign across TV and online to coincide with The Oscars. It shows the heritage that Rolex had in Hollywood. It surprised me that Rolex felt the need to do this

    More luxury related content here.

    The Worst Mission Statement Of All Time – Medium – epic

    Siberian tigers take out a drone that had been harassing them

    A Bathing Ape (BAPE) have done a tie-in with the new King Kong film. Given the Vietnam era setting I think that they could have done so much more such as ‘combat Zippo’ lighters, embroidered jackets and fatigues – instead there’s a t-shirt. You can tell that Nigo is no longer behind the wheel over there.

    A BATHING APE®︎ X KONG:SKULL ISLAND #bape #kongskullisland #KongIsKing #キングコング映画

    A post shared by A BATHING APE® OFFICIAL (@bape_japan) on

    A really nice film on data featuring Faris Yakob

  • The Yahoo Data Breach Post

    2014 brought us a Yahoo data breach only disclosed now; it formally declared the breach to consumers on September 22. This isn’t the first large data breach breach that Yahoo! has had over the past few years just the largest.

    In 2012, there was a breach of 450,000+ identities back in 2012. Millions of identity records were apparently being sold by hackers in August 2016 that the media initially linked to the 2012 breach. It would be speculative to assume that the records for sale in August was part of the 2014 raid.

    The facts so far:

    • 500 million records were stolen by the hackers. Based on the latest active email account numbers disclosed for Yahoo! many of these accounts are inactive or forgotten
    • Some of the data was stored unencrypted
    • Yahoo! believes that it was a state sponsored actor, but it has offered no evidence to support this hypothesis. It would be a bigger reputational issue if it was ‘normal’ hackers or an organised crime group
    • There are wider security implications because the data included personal security questions

    The questions

    Vermont senator asked the following questions in a letter to Yahoo!:

    • When and how did Yahoo first learn that its users’ information may have been compromised?
    • Please provide a timeline detailing the nature of the breach, when and how it was discovered, when Yahoo notified law enforcement or other government authorities about the breach, and when Yahoo notified its customers. Press reports indicate the breach first occurred in 2014, but was not discovered until August of this year. If this is accurate, how could such a large intrusion of Yahoo’s systems have gone undetected?
    • What Yahoo accounts, services, or sister sites have been affected?
    • How many total users are affected? How were these users notified? What protection is Yahoo providing the 500 million Yahoo customers whose identities and personal information are now compromised?
    • What steps can consumers take to best protect the information that may have been compromised in the Yahoo breach?
    • What is Yahoo doing to prevent another breach in the future?
    • Has Yahoo changed its security protocols, and in what manner?
    • Did anyone in the U.S. government warn Yahoo of a possible hacking attempt by state-sponsored hackers or other bad actors? When was this warning issued?

    Added to this, shareholders and Verizon are likely to want to know:

    • Chain of events / timing on the discovery on the hack?
    • Has Yahoo! declared what it knew at the appropriate time?
    • Could Yahoo! be found negligent in their security precautions?
    • How will this impact the ongoing attrition in Yahoo! user numbers?

    Additional questions:

    • How does Yahoo! know that it was a state sponsored actor?
    • Was there really Yahoo! web being sold on the dark web in August?
    • Was that data from the 2014 cache?
    • How did they get in?

    More Yahoo! related content here.

    More information
    An Important Message About Yahoo User Security | Yahoo – Yahoo!’s official announcement
    UK Man Involved in 2012 Yahoo Hack Sentenced to Prison | Security Week
    Congressional Leaders Demand Answers on Yahoo Breach | Threat Post