Planet Engines: Plagger and Planet

2006-07-30T23:16:00-05:00

Planet engines are applications that aggregate RSS/Atom feeds and generate composite feeds as well as a website. The generated feeds typically include RSS, Atom, FOAF and OPML. Two popular open source planet engines are Planet and Plagger. I've used both to create planet-style websites and here are my observations:

Planet: This is a Python-based planet engine originally created for Planet Gnome and Planet Debian. It is super easy to to install as there's not even really an install to speak of, you just untar it into place. The issue with Planet is that it has limited functionality and uses a monolithic architecture that makes adding features difficult. For example, the code to generate the HTML and XML files is in a 953-line file called __init__.py. I hacked a feature on to Planet a while back and had to modify some methods and add a new one to __init__.py. It worked but didn't seem clean or very maintainable so I'm reluctant to add even more features at this point. Here are my notes on Installing Planet.
Plagger: This is a Perl-based planet engine written by Tatsuhiko Miyagawa who currently works for SixApart, the makers of MovableType, LiveJournal, TypePad and Vox. Although Plagger is a planet engine it does a lot more and is really a pluggable RSS/Atom feed aggregation framework/platform that supports building planet-style websites via one of its many plugins. Its plugin architecture supports the following types of plugins: Aggregator, Bundle, CustomFeed, Filter, Notify, Publish, Search, SmartFeed, Subscription and Widget. Planet sites are generated by Plagger::Plugin::Bundle::Planet. Plagger will also send aggregated feeds to your email account using Plagger::Plugin::Publish::Gmail. This is similiar to what Newspipe (Python-based) does but Newspipe only does RSS to email. Configuring Plagger is simple once you've installed it, but installation can be tricky since it has many CPAN dependencies. However, it's not too difficult and I've done it on a dedicated CentOS server as well as on a shared hosting account at Dreamhost using these Installing Plagger instructions.

If you are interested in running a planet-style feed aggregation site, I highly recommend Plagger since it's just as easy to configure but has many more capabilities and a better architecture for extensibility. Installation takes a bit longer but give it a shot, it's worth it.

Python - First Impressions from Hacking PlanetPlanet

2006-06-13T21:52:00-05:00

Over the weekend I got to use Python for the first time to add channel categories to Planet which can be seen in action at Planet MVC. My first impressions of Python is that the resulting code looks very clean. The syntax was also easy to pick up primarily due to the excellent docs site and because it reminded me of JavaScript.

The first problem I ran into was using the regex object's re.match which definately does not follow the Python philosophy of "There should be one -- and preferably only one -- obvious way to do it" because there's more than one way and re.match is not obvious because it works different than other languages. Python's re.match automatically starts from the begining of the string unlike regex engines for other languages (grep|sed|awk|perl|ruby). This results in the following behavior:

# matches - pattern is at start of string
m = re.match('b', 'bc')

# does not match - pattern is not at start of string
m = re.match('b', 'abc')

# matches - add the .* to match any leading characters
# but .* isn't needed after '.*b' to match 'c'
m = re.match('.*b', 'abc')

# matches - second .* matches 'c' but isn't needed
m = re.match('.*b.*', 'abc')

I was sticking to re.match because I needed the MatchObject to capture groups but it didn't DWIW. After posting to a forum, I learned that I should use re.search instead which will match anywhere in the string. re.search also returns a MatchObject but I didn't expect this at first because search != match. I get the impression re.match and MatchObject were created first and re.search was added later. Perhaps the MatchObject should be called something generic, like ResultObject, in the docs. I'm sort of curious but when is re.match desired over re.search? It seems like re.match is useful only in a limited number of situations and then it's always easy to implement it with re.search.

The open issue for me is Python's dependency on indenting. Lately I've been using tabs exclusively for indentation because I like to see the tickmarks may editors show for tabs. This lets me quickly know what level I'm at. This python-list thread was titled Tab Wars and argued in favor of spaces over tabs. Planet is indented with spaces and that's what I used for my hacking. Given my preference for tabs, the more time I spend with Python the more likely I'll switch to tabs. Do I need to convert the entire script to tabs or can I mix tabs with spaces? What do people with a preference for tabs do with Python code that uses spaces?

I've been thinking about releasing my enhanced version of PlanetPlanet which allows one planet site to run multiple planets but now I'm thinking I want to do more enhancements to make it even more user friendly. Let me know how Planet MVC is working for you or if I can add any additional blogs. I've already found it pretty useful to find out about things I otherwise wouldn't have.

Here are some notes on getting Planet running.

Dev411 Blog: Category planet

Planet Engines: Plagger and Planet

Python - First Impressions from Hacking PlanetPlanet