RSS-Spider

Been setting up a WAMP server on the new Vista laptop for the past couple days and haven’t gotten it quite right yet. My previous laptop had XP and seperate installs of Apache 2.(something) MySql 5 and PHP 5. It took a bit of time to get the system to my liking, but once I found out that I could do virutal hosts on a laptop I was loving it.

Originally the MSI install package from the Apache website would not install no matter what I did. Even tried the MSIEXEC fix I stumbled on and that didn’t work. Google searched and it seemed like everyone was having the same problem but nobody had a good solution. I kept seeing “try WAMP SERVER”. I’d seen XAMMP before and wasn’t all that impressed with it, but hey I need Apache, PHP & MySql on my machine to do my job… so why not. Downloaded it and it installed nicely the first time without out errors or roll back.

The only problem I then ran into is that Virtual Hosts weren’t working at all. Any time I’d ad something like

ServerName spider
DocumentRoot “C:/web/www.rss-spider.com/html”

I’d get an error.

Actually they all taste like burning…

Last night we experienced an 8 hour outage during our weekly database backup & purge.  The database of headlines has grown so massive that it took way too long to purge out any old & “spammy” headlines.  The last time we did this purge was back in December and we only experienced a 15 minute outtage.  I was up till 6 am waiting for everything to finish.  If you had visited the site during this time you would have been directed over to the my.rss-spider.com page which was incorrectly listed as our forwarding page.  My.RSS-Spider.com is a project we’re working on which is in it’s infancy.  In a nut shell what it will do is allow users to have their own webspace to aggrogate RSS feeds from RSS Spider.  So instead of coming and searching for something every time you can simply go to your my.RSS-Spider.com home page and view all the new feeds matching your search criteria.  Again.. this is something thats way down the road, however, we are planning to have a Beta release in late March.

I’ve been noticing that some times it takes upwards of a minute or more to return a search. This ain’t good. Looking into it I see that MySql is attemping to do UPDATES and SELECTS at the same time. The spider is updating the RSS feed links to current time (last polled) while people are using the database. I’ve bundled all the updates into a single querey and added PRIORITY_LOW to the updates. Remove all other updates which weren’t doing anything meaningful ie: number of times an item was viewed on a returning search page… bah… crap…