Dev411 Blog

There is a lot of choice on the CPAN for open source Perl libraries and sometimes it's difficult to get an idea of how modules compare to each other. CPAN Ratings is a good source of reviews but it's not convenient to compare one module with another. To provide a partial solution, I whipped up a quick CPAN Compare page which will pull the CPAN Ratings from a number of modules and summarize them for you.

I've worked on a number of database-driven projects and no matter how much people want database abstraction, it was always difficult to code and maintain. I was recently reminded of this when I read this Drupal article on dropping PostgreSQL support. Not only can it be difficult to maintain support for multiple databases, but it may be difficult to find developers.

One solution of modern programming is to move database abstraction from the code to the infrastructure using a ORM (Object-Relational Mapper) or Data Mapper. A ORM and Data Mapper abstracts the database for you so you no longer have to do tie db abstraction to each app. Not only does it let you code once for multiple databases it lets your users migrate their data from one database to another. This blog runs Typo which is based on Ruby on Rails and ActiveRecord. I've been contemplating migrating Typo from MySQL to PostgreSQL and I've been told that it would be as simple as exporting the data with YAML, updating the database.yml file and importing the data. I haven't gotten around to doing it yet but it is a powerful idea. ActiveRecord is a data mapper and isn't as flexible as a full blown ORM but it gets the job done for the most part. For a full-blown ORM, I think of Perl's DBIx::Class which provides a full OO interface to the RDBMS allowing you to code just once for multiple DBs without limiting you when you want to use some esoteric database-specific SQL. DBIx::Class is often used with the Catalyst Framework but is also used by itself.

There are PHP frameworks out there like Symfony and Cake but do any of them have stand-alone ORMs? If so, could Drupal move to something like that and solve their maintainership problems once and for all? Drupal is part of the Go PHP5 effort so there should be no issue using PHP 5 OO. Something to think about for the Drupal folks if a PHP ORM is available.

One of the mysteries of Perl to me is that why, as of yet, is there no UTF-8 support in DBD::mysql although this issue has been discussed on the msql-mysql-modules list since at least 2003 (using the MARC archives). This is also given that MySQL does have UTF-8 support itself.

I recently looked at using various encodings for hashed UIDs, e.g. UIDs generated by a crytographic hash algorithm such as SHA-1 or MD5. These are often useful when the UID does not need to have human meaning but should exhibit some uniformity, such as character set and length. I considered Base64 and hexadecimal first because they are commonly used by crypto libraries and then decided on Base64 and Base32 where appropriate. Base36 is actually the most compact case insensitive encoding (using Arabic numbers and Roman letters) but is not an option for me at the moment because there's no Perl module for it that will take arbitrary text and binary input at the moment. Math::Base36 exists but only handles numbers.

Perl has two UTF-8 encodings, utf8 which is Perl's liberal version and UTF-8 which is a strict interpretation, aka utf-8-strict. The liberal version allows for encoded characters outside the UTF-8 character set, however you can run into problems when interoperating with applications that expect utf-8-strict, such as PostgreSQL. Here's a function I wrote to strictify utf8 to UTF-8 using the Encode core module:

use Encode;

sub strictify_utf8 {
    my $data = shift;
    if (Encode::is_utf8($data) && !Encode::is_utf8($data,1)) {
        Encode::_utf8_off($data);
        Encode::from_to($data, 'utf8', 'UTF-8');
        Encode::_utf8_on($data);
    }
    return $data;
}

I recently responded to someone asking how to get a Unicode hex codepoint from a Unicode literal on DevShed Forums. Since I think it may be more generally useful, here's my solution. The following function takes a unicode literal, converts it to a decimal representation using unpack and then converts it to hex usning sprintf:

sub codepoint_hex {
    if (my $char = shift) {
        return sprintf '%2.2x', unpack('U0U*', $char);
    }
}

my $cp = codepoint_hex('カ'); # eq '30ab'

Whenever popular Perl sites are discussed, a number of large websites are mentioned with the usual suspects being Amazon, IMDB, LiveJournal, Slashdot and others. While this is a good list, what often doesn't get mentioned is the new Web 2.0 sites that Perl 5 is powering. Perl 5 web frameworks (such as Catalyst and Jifty), OO models (Moose) and other techniques (PAR) dramatically improve Perl for web development and large-scale projects making many complaints about the language irrelevant. These, when combined with CPAN, continue to make Perl 5 a very attractive language for new web apps.

Examples of Perl-powered Web 2.0 sites include del.icio.us, (which uses the Mason templating system / mini-framework), EditGrid (an AJAX-enhanced online spreadsheet running on the Catalyst framework) and Vox (a multi-user blogging site by Six Apart, maker of LiveJournal, MovableType and TypePad). Other sites include HiveMinder, IUseThis, and MighTyV. Of these six sites, four are running on the Catalyst framework. HiveMinder is built on the Jifty framework. Yahoo is even interested in developers with Perl/Catalyst skills along with C and Java for this Senior Software Engineering position (screenshot from jobs.perl.org). Screenshots and links of active sites are provided below.

del.icio.us Description: Social Bookmarking Perl Framework: Mason

EditGrid Description: Online Spreadsheets Perl Framework: Catalyst

Bill Odom, President of The Perl Foundation, recently posted a blog entry called TPF RFC. My comment to that blog entry wasn't approved for some reason so I figured I'd post some easy-to-accomplish recommendations here.

Lucene is a Java-based fulltext indexing and search solution run under the Apache Foundation. It is arguably the most popular of many fulltext solutions now but its use of Java makes it a secondary choice for many non-Java projects. To improve the speed of Lucene, there are several C/C++ ports including:

• Lucene4c: Apache Foundation project
• Lucy: Apache Foundation project
• CLucene: Sourceforge project

I've been looking for a program that will take full screen shots of web pages even when the web page is larger than the window size on my physical screen, requiring scrolling. This morning I found such a program in Petr Šmejkal's Win32::CaptureIE when it was mentioned by Displeaser on DevShed Forums in the "Screenshot of webpage" thread. It uses ImageMagick for image manipulation.