The Long Tail
I’ve just read a book called The Long Tail which in a nut shell explains how online properties like iTunes, Amazon & others are selling more very niche items. The term long tail comes from the what the graph looks like when you graph out the sales of say Music or Books online. With the every expanding selection of items online the tail gets longer and longer. Take for example Netflix. Netflix has thousands upon thousands of DVDs for rent on it’s site. A handful of those DVD’s are what we commonly think of as blockbuster videos… ie X-Men, Harry Potter, Capote etc. However, these releases represent a small portion of what Netflix is renting. There is a HUGE amount of movies which never made it to your local cinema plex or to the local Hollywood Video store. They never made it there becuase of a lack of shelf space. Shelf space is hugely expensive and there for reserved to what the video store “thinks” it can make the most money on. Whats on the shelf represents a FRACTION (maybe 2%) of whats really out on the market in any given year. The rest of the non economically successful DVDs never show up in your local video store. However, Netflix doesn’t have to worry about shelf space or store overhead or other expensive things, so they can afford to buy & rent out movies which never saw the inside of a thearter or Blockbuster Video.
Now conventional wisdom would say that block buster movies are block busters becuase they’re GREAT movies and that if a movie wasn’t a block buster then it’s not that good. Totally wrong. A movie is a block buster becuase some studio execs gave it the “green light” to get promoted and pushed out onto the movie going public. There are only about 100 hollywood movies that come out each year. Each one of these has a budget for promotion & theather runs. A movie to be commercially successful must pull in several thousand people during its run at a theather. Movies that don’t or movies that studio execs don’t think will have the draw dont get promoted, or even made by the big studios. Futher more movies which get standing ovations at Sundance might not go any further than that.
Now the long tail comes into play with a huge selection and a good post filter. One of Netflix’s features that I like the most is the RIFL or “recomended if you liked” option. After initially signing up for Netflix I was told that I should go though an rate different movies on a scale of 1 to 10. From that Netflix would be able to pick movies that it thinks I might like. The more movies I rate the more likely Netflix is to pick something I like.
SOOOOOOOOO Whats this got to do with RSS-SPider? Well one of the first things I noticed shortly after launching the site was the long tail effect in searches. A lot of people look for the same top 100 things, but an even greater amount of people are looking for a lot of other different things.
Note had I not installed the newest version of jpgraph on this server I’d be able to graph something out. However, it’s looking for a newer version of php and we’re not planning on upgraded for a few weeks 
I’ve been reading a book I got for Christmas by Thomas A Limoncelli called Time Management for System Administrators. I’ve found it totally engrossing. So far in the 3 chapters that I’ve read I’ve been able to clear out the clutter in my office at my “real job”, start to focus on priorities a little better, and I’m even tempted to start using my Palm Pilot again. Now if I could only get everyone else in my office to read this book. Our research directory is constantlly frazzled… our production people are always on edge, and our sales reps are always in foul moods.