Dev411 Blog: Category scalability

Displaying Dates and Times Using JavaScript

2007-02-05T18:28:00-06:00

Some considerations when displaying dates and times on a website include showing delta times, customized timezones and caching. Often it's nice to show a delta time like "10 minutes ago" or "5 days ago" to give readers a frame of reference instead of an absolute date. When the date is far enough in the past and an absolute date becomes desired, customizing the date to the user's timezone is useful. And if your site grows large enough that caching becomes useful, finding a way to display customized deltas and timezone information in a cacheable static page becomes an ideal solution.

JavaScript is an ideal solution for all three issues. With JavaScript you can place an absolute date in the web page and have the JS dynamically update it when the page is loaded. This can be used to calculate delta times and accommodate timezones as well. The result is that the page can embed the same date every time and thus becomes more cache-friendly.

The Typo blog engine (which runs this blog) comes with a useful MIT-licensed JavaScript in it's typo.js script. Just copy three of the JS date/time functions, wrap your dates with spans (using the appropriate class name and absolute date in the span title) and then call show_dates_as_local_time() when your page is finished loading. The two other functions you'll need are get_local_time_for_date(time) and distance_of_time_in_words(minutes). This is what I did for Planet Catalyst's Plagger theme a while back.

Although it's pretty easy to accommodate timezones, the Typo script doesn't do that. I've done this for some projects and might post some code in the future but it's not hard.

Customization and cacheability, two great advantages for using JavaScript to handle dates and times.

Planning Ahead for Open Source Storage Scaling

2006-10-24T11:56:00-05:00

Recently eWeek ran an article on eHarmony's storage scaling solution choice which discussed how they chose to go with proprietary solutions from 3PAR and ONStor. I was hoping to learn something interesting about their deployment architecture but the most interesting things I learned was that eHarmony has 8+ million users, 9+ million photos and their proprietary solution vendor choice. Some interesting quotes from Mark Douglas, eHarmony's VP of Technology:

"We find ourselves having to buy storage about every 90 days."
"The other solutions we considered had a learning curve and a level of complexity that we just didn't want to undertake."
"There was going to be a lot of hands-on work to do with our six years' worth of data. We wanted a more automated system, for sure."

It seems like what happened is that they didn't plan for growth and by the time it hit them they were too busy and didn't want to deal with it. Going with proprietary solutions seemed like the easy way out. However, one has to wonder if relying on proprietary solutions is a good decision for further scaling needs. In Steve Bryant's article "Top 10 Reasons It's Almost Impossible to Compete with Google" he lists distributed infrastructure as the very first reason:

Huge, Distributed Infrastructure -- The obvious advantage is Google's huge infrastructure, which is distributed across 450,000+ servers across the globe. By distributing its infrastructure, Google decreases router and switch delays and delivers faster performance to its worldwide users. Not only is search faster, but products work better too.

One thing that has been popularized about Google is their massive use of cheap, commodity hardware, not large proprietary systems like those that 3PAR and OnStor seem to build. While Google uses the closed-source GoogleFS, there are some similar FOSS solutions, namely SixApart/Danga's MogileFS. MogileFS was built for LiveJournal because the alternatives were, according to Brad Fitzpatrick's 2005 OSCON presentation (pdf):

closed, non-existent, expensive, in development, complicated, ...
scary/impossible when it came to data recovery

The PDF presentation is a bit long at 80 slides b/c it covers all of LiveJournal. I've extracted the MogileFS slides which is just 13 slides to give you an overview. If you are interested, it does make sense to read the full presentation because it also goes over Perlbal and memcached. One great thing about MogileFS is the automatic ability to make multiple backup copies means that RAID and tape backup are not required. A big difference compared to the "big iron" solutions that 3PAR and ONStor seem to provide. SixApart continues to support MogileFS as an FOSS project and recently held a MogileFS Users/Developers Summit at their San Francisco headquarters. Although MogileFS is the primary, stable and proven FOSS DFS at the moment, there are others in development, including the Hadoop DFS which is part of the Apache Lucence project.

The interesting thing about eHarmony's choice is that MogileFS is a free open-source solution that can more than fulfill their needs. LiveJournal has 60+ million images comprising 6-7TB of information stored in their MogileFS. A little forward planning can help your site scale storage without having to rely on proprietary solutions.

MySQL Deployment Presentations

2006-10-05T22:07:00-05:00

I just ran across the MySQL Web 2.0 page which lists a number of their users including the following:

The most interesting thing from that page, however, is links to various presentations given by those sites on how they architected their sites to scale with MySQL, some of them scaling up to hundreds of MySQL servers.

I've included a list of the presentations below because you have to do some page hopping to find which customer pages actually have presentations. The presentations are in PDF and PPT which I've indicated. None of the presentations are in XUL nor do any use the Takahashi method, both of which are becoming more popular.

eVite 2005-04-19 pdf
Flickr 2005-10-18 ppt
Flickr 2006-03-14 pdf: ~25,000 db transactions/second peak
LiveJournal 2004-04-01 pdf
LiveJournal 2005-08-04 pdf
Mixi 2006-04-26 pdf: More than 100 MySQL servers; Add more than 10 servers/month
Technorati 2006-04-26 ppt
TicketMaster 2005-09-05 pdf
Wikipedia 2006-04-26 pdf: >25000 SQL requests per second

I wonder if there's anything similar for PostgreSQL deployments at high traffic sites. Let me know if there are other deployment presentations for MySQL, PosgreSQL or other databases at well known, high traffic sites.

Update: Lukas Kahwe Smith gave me a link to OmniTI's PostgreSQL deployment presentation which discusses their migration from Oracle.

OmniTI pdf

Update 2: Some additional information on MySQL and UUID is available on articles by FeedLounge and Brad Fitzpatrick (chief architect for SixApart). Interesting discussion on how InnoDB clusters by PK.

Update 3: Robert Ficcaglia gave me this link to the Wikimedia Servers page at meta.wikimedia.org. Looks like good reading!

Hadoop Distributed File System

2006-08-23T00:49:00-05:00

I just ran across the Hadoop DFS which is an open source alternative to distributed file systems such as GoogleFS, OneFS and others. GoogleFS and OneFS are both proprietary so it's nice to finally have a FOSS solution. MySpace uses OneFS. From the Hadoop Wiki:

Hadoop's Distributed File System is designed to reliably store very large files across machines in a large cluster. It is inspired by the Google File System. Hadoop DFS stores each file as a sequence of blocks, all blocks in a file except the last block are the same size. Blocks belonging to a file are replicated for fault tolerance. The block size and replication factor are configurable per file. Files in HDFS are "write once" and have strictly one writer at any time.

Until now, I had only been aware of MogileFS for FOSS solutions, however MogileFS is designed for smaller files such as images and the others are designed for very large files. It will be interesting to see how much traction Hadoop DFS gets since it could be very useful and a good FOSS compliment to MogileFS. Hadoop is part of the Lucene Apache project.