The Long Tail

There's been some debate about the effect of the web on commerce, whether access to "the long tail" would be a boon or not. Recently, thinking seems to have shifted towards the "winner takes all" theory, that the best-selling whatever, by reason of being best-selling, would find itself cemented as the best-seller by the use of search engines, user recommendations and merchandizing software that looks at preferences and suggests what to buy.

This article in Techdirt, Winner Takes All, Long Tails And The Fractilization Of Culture looks at the argument for winner takes all and concludes that things don't work that way - that winners (and tails) are fractal. A big winner and long tail for the whole world, and similar smaller winner and long tail for every niche.

Just as a fractal repeats its same pattern as you zoom in and look closer on the smaller segments, so do cultural subsegments. And those segments continue to thrive, despite the recommendation systems just pushing people to the hits. Part of that may be that once you've begun exploring those subcultures, the recommendation engines and collaborative filters drive you towards the "hits within" the subculture.

This seems intuitively true to me. Whoever's on top of music today (and I have no idea who that is - Kanye West? Taylor Swift?) will get the most hits on any generic search, but who searches generically? I'm always looking from within my subculture, and so are you.

A trainspotter, going to his trainspotter message boards and reading trainspotter sites, will always find that Mr. Big Trainspotter is more important than Kanye West in his world, and if you do furniture restoration, French polish is always going to come up more often in your searches than Miller Lite beer. The idea that search engines and preference engines drive us all into the mainstream is intrinsically flawed. The web's Killer App - search engines - actually dig us deeper into our own little world with every search.

Now, that's not to say everything's fine. In a way, that might be a worse scenario. Almost every search I do proves I'm right, because the keywords I use lead me to sites that agree with me. If I find 60 sites that agree with me on something, I have no idea whether sites that disagree number 6, 60, or six million. All I know is somebody agrees. Fragmentation of this kind looks to me as though it will be a bigger problem than homogenization.