RSS-Spider

Yesterday marked the launch of the Hot Words section of RSS-Spider. What this section does is mash up all the posts from any given day, sort all the words from that day and count the number of times any specific word appears. From there it takes the top 100 words as they appear and rank them in font size order.  So on January 25th the system processed all the documents in the database that had a pubdate of January 24 (this date comes from the RSS feed that the spider pulled) and found that in the top 100 terms used on that day Alito, Bush, Iraq and War all came up…

Clicking on any one of these terms will pull up all the articles stored in the database for that day with that term.

Currently we are only processing English language feeds, but our next step is to add a Hot Words for German users.

If you want to see what items are hot in the RSS feeds we’re polling check out the WhatsHotYesterday.php link under the Beta section.  What we’re doing here is taking all the feeds we’ve spidered that have a post date of yesterday (what ever that might be) and mashing the headlines and bodies together sorting out all the words then figuring out which words appeare the most.

So far we’ve see Alito pop up a few times on our Friday test, as well as Bush.  Guitar seems to be a big one.  Right now it’s only going to return blogs where the language is EN or english.

Soon we should have a whats hot yesterday database of every pubdate in our database.

First Happy New Year to the both of you who read this page. Welcome 2006. May it be a great year.

Ok… now that thats out of the way our first big accomplishment of the year with RSS-Spider is to create a search bar plugin for Firefox. I had noticed some traffic coming from an odd Mozilla page tonight and see that Xavier Casto had created a search bar plug in for RSS-Spider. Cool… however, Firefox will only do GETS to search engines and RSS-Spider will only do POSTS. So Xavier created a special page somewhere on WISC.edu which would convert the GETS to POSTS. Well if WISC.edu goes down this isn’t much help and the plugin dies on the vine… or bar.

Now to avoid this I tooled around for about an hour reading the documentation on creating plugins and figured that I could easily create a GET->POST page. Ding ding ding. Well if you would now like to add our PLUGIN to your Firefox browser go to this page…

ADD RSS-SPIDER SEARCH BAR PLUGIN TO FIREFOX

Click on the link where prompted and you’ll then notice the RSS icon in the drop down search bar in the upper right hand corner of Firefox.

Happy New Year!

Building a directory. Currently it resides at

http://www.rss-spider.com/related

In it will be listed RSS & other tid-bits useful to anyone looking to build an RSS fee or blog etc. Also working on creating RSS feeds for search. Didn’t dawn on me until yesterday that people might want to link their searchs into the RSS readers. Duh. Example of what I’m talking about here…

http://www.rss-spider.com/rss.php?TM=CLEVELAND%20BROWNS

Must fix head in the XML stream. It’s not an application.