Fatal flaw in the new Feedster 500

Jason Calacanis was right in claiming that there is no good method for accurately representing blog popularity or influence.  He called the blog search sites to the carpet and offered $50,000 in advertising on the Weblogs Inc Network (WIN) for the first search site to come up with a scheme that is better than the current inaccurate system employed by Technorati.  Feedster shot to the front and have published their admittedly preliminary Feedster 500 list that is based on the count of links to the blog in question.  It’s not a bad list but while really looking at the blogs on this list one fact jumped out at me.  Blogs that belong to a network are driven up the list because other blogs in the network cross-link like crazy.  You can see this on Weblogs Inc. blogs all the time as they are frequently running “best of WIN” posts across the board that link to other blogs on the network, groups of them at a time.  These links are apparently counted by Feedster making these blogs appear to have many more links than it would show if they didn’t do this cross-linking.

I do two podcasts for The Podcast Network which also has a network of blogs, one for each show, and it surprisingly shows up high in the Feedster 500 list.  While it makes me quite happy to see it appear on the list am pretty sure it is because all the different network blogs have multiple links back to the main one.  It’s the same phenomenon we see with the WIN blogs and even some of the Gawker ones.

I am not saying these blogs in the top 500 list should not be there, I am just pointing out the methodology is highly flawed.  Feedster should not count links to any blog from other blogs on the same domain if they want to prevent inter-network cross-linking from driving the number of links to any given blog inaccurately high.  This is the same principle that spammers are using to skew Technorati.  I would love to hear what others think about this issue.

Comments

[Scott Rafer]

Thank you for making the issue this clear. We'd heard similiar items from other sources but had not yet had time to sit down and detail it out for ourselves. Now we don't have to.

I'm re-posting part of this on the public algorithm wiki at:
http://www.socialtext.net/topicindex/index.cgi?feedster_top_500

If you want to drive us more on the detail, please add to the wiki. Thank you.

[Jason]

I think having an "indie" index would be a great idea... basically take out all the Gawker, WeblogsInc, etc. blogs.... I'm all for it because the list of indie blogs is where we find new bloggers to work with!

[ZuDfunck]

The Measurement of Blogs, Websites, Podcasts & Vlogs will be forever corrupt. Be it for technical issues as you mention here or the jaded nature of those wanting to be Measured!

I therefore think it's important to not engage in the activity at all. Focus on your content make friends where you can and know you are alright with the world no matter what some list may say.

[Michael]

As I see it the same problem exists with blog 'alliances'. Just take a look at the difference in the Truth Laid Bear's ecosystem rankings based on links vs. their ranking by traffic.

And speaking of traffic, I'd love to see a blog's traffic taken into account. Some of us, ahem, may not get many links from other bloggers but have a pretty good level of traffic. Of course the problem here is deciding on how to measure the traffic...