When is an RSS item New? - The RSS Blog
RSS, OPML and the XML platform.
Copyright 2012 World Readable
The RSS Blog
<< Previous Main Next >>
Sat, 07 Apr 2007 19:49:31 GMT
When is an RSS item New?

This last week, I pointed to some flawed thinking about RSS. In particular, about how to determine if an <item> in an RSS feed is new content. Today, I realized that I pointed to the flaw, but didn't tell you how to do it better. The author suggested using <pubDate> as an indicator of new content. Of course, this doesn't always work as some RSS items update their pubDate when the item is modified. The best approach is to use the <guid> element, but even the guid element is optional. With all these optional fields and with RSS 1.0 and multiple version of Atom to worry about, just what is a developer to do? What I did was write a small little function that either returns the guid from an RSS item, or derives one from the remaining content. The algorithm is really simple, this is Really Simple Syndication. Here's the pseudo code:

if (exist(item.guid)) return item.guid.text;
if (exist(item.link)) return item.link.text;
if (exist(item.pubDate)) return item.pubDate.text;
if (exist(item.title)) return item.title.text;
if (exist(item.description)) return hash(item.description.text);
if (exist(rss10:item.link)) return rss10:item.link.text;
if (exist(atom03:entry.id)) return atom03:entry.id.text;
if (exist(atom10:entry.id)) return atom10:entry.id.text;

Since an RSS item must have at least one of <link>, <title> and <description>, you can always derive a unique id for RSS feeds. Now it's simply a matter of comparing the result of this function to the results of previously sent items and you know if this item is new content or not.

Please leave a comment, if you have ideas on how to make this better. I'm gonna try to write more information articles like this one going forward.

Reader Comments Subscribe
Thanks Nick! That's some great feedback based on real experience. I like the idea of concatenating the strings and will likely change Rmail to do the same.

Type "339":
Top Articles
  1. Unblock MySpace
  2. MySpace
  3. FaceParty, the British MySpace
  4. del.icio.us and sex.com
  5. Blocking Facebook and MySpace
  1. Review of RSS Readers
  2. MySpace Layouts
  3. RSS Stock Ticker
  4. RSS Gets an Enema
  5. Google Reader rejects del.icio.us