eriksmartt.com/blog

tracking software releases

I use Versiontracker, MacUpdate, freshmeat, etc. to watch for software updates. Freshmeat and Versiontracker offer RSS feeds to make things a little easier, but other sites require daily, manual visiting to check for updates and new releases. That sucks. In fact, keeping up with patches and new releases is such a hassle that Versiontracker and MacUpdate have even developed business models around this annoyance by offering paid services utilizing desktop tools to monitor your currently installed software for updates.

While I like the idea of using a background application to monitor this stuff (meaning, I like the idea of automating the task), I don’t like the service being bound to one particular tracker. For example, even if I did pay for the Versiontracker service, it still wouldn’t monitor all the unix and open-source stuff I run. What I need instead, is a generic way to monitor software projects for updates — and I want it to come to me via RSS/Atom; So naturally, I built a simple proof-of-concept tool that pulls RSS feeds from software trackers, compares the contents to the list of software I’m interested in, then generates a different RSS feed containing ONLY the updates I’m interested in. Making it work turned out to be rather easy.

To start with, I focused on the Versiontracker RSS feed. The folks at Versiontracker were nice enough to use a URL scheme that gives each software product a unique ID, making it very easy to parse. The feed is updated rather frequently though (and missing a post would be bad), so I needed a way to continuously poll their feed and cache the results without being a bad netizen. The solution? Bloglines, of course. Adding the feed to Bloglines let’s me catch all the content updates without worry. And since Bloglines offers an API for pulling content, it was simple enough to write a script that checks the feed. The meat of it looks like this:

1: # Connect to Bloglines.
2: bws = BloglinesWebServices("***@***.***", "****")
3:
4: # Fetch the Versiontracker feed and mark it as being read.
5: data = bws.getitems(*****, True)
6:
7: # If we got something...
8: if len(data.entries):
9:    # Compare the VT feed to my saved list of software
10:   for i in data.entries:
11:       for key in self.listing.entries():
12:           if i.link.endswith(unicode(key)):
13:               # Save a list of dictionary entries that describe the software.
14:               self.updated.append({'title':i.title, 'link':i.link, 'description':i.description})

The BloglinesWebServices class is defined by pybloglines, which is built on-top of the universal feed parser.

In line 11 you’ll see that I’m pulling from a data structure called listing.entries. This is just a list of ID’s that are pulled from a text file. Building the text file is fun though — all it took was a simple CGI and a Bookmarklet. Using the Versiontracker site, I navigate to a piece of software that I’m interested in and simply click my Bookmarketlet to pass the URL to a CGI. The CGI extracts the Versiontracker ID from the URL and stores it in the text file!

The rest of the code takes the data structure `updated.append` (from line 14) and generates an RSS feed from it. Then, in a nice piece of circular processing, this new RSS feed was also added to my Bloglines roll. With both feeds in Bloglines, the content monitoring is automatic from my perspective, with the whole system flowing like this:

1. Bloglines polls Versiontracker for updates and caches the results.
2. Bloglines polls my content filtering CGI for updates.
3. The CGI authenticates with Bloglines.
4. The CGI then pulls the cached contents from the Versiontracker feed (marking it all as read.)
5. The CGI returns an RSS feed containing only the items that I’m interested.
6. Bloglines caches the results of my CGI processing for the next time I log in.

Beautiful.