There and Back Again

I have a lot more change's i'd like to make to webthumb so the last article (Adding AJAX to a website step by step) is now the start of the series. In this article will be taking front page of webthumb and making it live. The the stats and recent thumbnails will update in real time, making for a nice slick presentation.

I hope to release and article for each improvement I make to webthumb, giving everyone idea of whats involved in AJAXing a real website. But a series of articles won't cover everything you want to know about AJAX. If your interested in learning more you should take a look at my book, Understanding AJAX (should be shipping september 1st), it covers everything from basic AJAX implementations to usability and debugging.

Getting realtime data with AJAX

The frontpage of webthumb could be updated with each new thumbnail that is created. In straight HTML you might handle updating things by using a meta refresh. The problem with this, is your loading a lot of repetitive content, and the full page refresh makes for a clunkly look.

With AJAX you can poll the server on a regular interval and just update things when data has changed. Polling approaches can cause scalability problems since they can create a large number of requests to your server. Hopefully this won't be a problem in this case since i'll be making small requests and returning a large amount of data, and I don't need to make them that often. There is also an alternate approach that you can use called comet, that works by each client keeping a connection open to your server and pushing data to them. But a push approach brings about a whole new set of scaling problems, so will stick with tried and true polling, and i'll cover comet some time in the future.

Exporting the PHP Class

Like the last upgrade will be using HTML_AJAX for our AJAX communications. On the server side I created a new class called WebthumbStats, it has a single method named "updatesSince", that takes a timestamp as its parameter. If you were using it in php you'd get something that looks like:

<?php

$stats = new WebthumbStats();
$update = $stats->updatesSince($lastPoll);

?>

If nothing had changed since my last poll i'll get a timestamp back. If something has changed I'll get an array of data back containing the basics stats, a timestamp, and the new image ids. Will use the timestamps from the server when making the next poll request that way we don't have to worry about problems with the client's time settings messing things up. A result might look something like:

<?php

array(
  'queue' => 1,
  'lifetime' => 1324,
  'today' => 329,
  'time' => '2006-08-07 16:45:01',
  thumbs => array(
    array (
      'img' => '/webthumb/data/ba86/wt44dac13bcba86-thumb_medium.jpg',
      'url' => 'http://bluga.net',
    ),
    array (
      'img' => '/webthumb/data/46d6/wt44dac3c4546d6-thumb_medium.jpg',
      'url' => 'http://blog.joshuaeichorn.com',
    )
  )
);

?>

After writing the PHP class I wrote a small test script in PHP and made sure it was working right, then I registered it in ajax.php.

<?php

require_once 'HTML/AJAX/Server.php';

class WebthumbServer extends HTML_AJAX_Server {
        var $initMethods = true;

        function initRequestStatus() {
                require_once 'lib/RequestStatus.class.php';
                $this->ajax->registerClass(new RequestStatus(),'RequestStatus',array('getStatus'));
        }

        function initWebthumbStats() {
                require_once 'lib/WebthumbStats.class.php';
                $this->ajax->registerClass(new WebthumbStats(),'WebthumbStats',array('updatesSince'));
        }
}

$server = new WebthumbServer();

$server->handleRequest();

?>

You'll notice the basic registration is the same style as we used for RequestStatus but now its enclosed in a class. The init method functionality in HTML_AJAX_Server gives you a way to organize how the PHP class are setup for AJAX access. It also means that when a request is made to RequestStatus WebthumbStats code won't be loaded. With just two classes this doesn't save a ton of overhead, but every lit bit helps, and when you get too 20 or 30 AJAX classes it can make a huge difference.

Polling with HTML_AJAX

The JavaScript side is really simple. We poll on a set interval, with a check to make sure we aren't waiting on another poll to finish. Other then that its just a matter of getting a starting timestamp from the server when we generate the page. The polling code looks like:

HTML_AJAX.defaultServerUrl = 'ajax.php';
var inProgress = false;
var statsInterval;
var lastPoll = '< ?php echo $now; ?>';

function setupStats() {
        statusInterval = window.setInterval(statsUpdate,8000);
}
function statsUpdate() {
        if (inProgress) {
                return;
        }

        inProgress = true;

        HTML_AJAX.call('WebthumbStats','updatesSince',statsCallback,lastPoll);
}
function statsCallback(result) {
        inProgress = false;
        if (result.time) {
                // do something with updates here
                lastPoll = result.time;
        }
        else {
                lastPoll = result;
        }
}

Updating contents on new data

Lets start with updating the basic stats since thats the easy part. We just give the div containing each of the statistics an ID and then replace there innerHTML with the new content. This code looks looks like:

$('queue').innerHTML = result.queue;
$('lifetime').innerHTML = result.lifetime;
$('today').innerHTML = result.today;

Updating the thumbnails is a little harder since we want to add new thumbs on the left of the list and drop the old ones off the right. The thumbs array we have returned is shown oldest first, so we can just append to the begining of the list and delete from the end. Using a little getElementsByTagName, insertBefore and remove child lets us make the image updaes a reality.

                var images = $('recent').getElementsByTagName('div');

                for(var i = 0; i < result.thumbs.length; i++) {
                        var newImg = images[0].cloneNode(true);
                        newImg.getElementsByTagName('img')[0].src = result.thumbs[i].img;
                        newImg.getElementsByTagName('a')[0].href = 'pickup.php?id='+result.thumbs[i].id;
                        newImg.getElementsByTagName('a')[1].href = result.thumbs[i].url;
                        newImg.getElementsByTagName('a')[1].innerHTML = HTML_AJAX_Util.baseURL(result.thumbs[i].url.substring(7));

                        $('recent').insertBefore(newImg,images[0]);
                        $('recent').removeChild(images[4]);
                }

The line using HTML_AJAX_Util.baseURL is updating the domain below the image, were using baseURL just like you would use basename in PHP. This domain chopping keeps long urls from making the list look bad.

Using HTML_AJAX to manage JavaScript library delivery

HTML_AJAX delivers its JavaScript libraries through HTML_AJAX_Server. This code can also be used to deliver any other JavaScript library you want. Its main features are, setting headers for client side caching, and allowing you to easily combine multiple files together so clients don't have to open as many connections to your server.

To finish this update, I'm going to need moo.fx so my first task is too register it with my server.

<?php

$server->registerJsLibrary('moofx',array('prototype.lite.js','moo.fx.js'),'/home/htdocs/javascript/');

?>

This registration makes the keyword moofx delivery the prototype lite depenency and the moo.fx library. These files are grabbed from /home/htdocs/javascript on my server.

Then i just update my include to pull in moo.fx

<script type="text/javascript" src="ajax.php?client=html_ajax_lite,alias,moofx"></script>

Adding some Effects

The auto updating stats are neat, but the change is so subtle people might not notice its happening. To make it clear an AJAX update is happening I'll add a couple effects.

I'm going to animate the height down to zero on a thumbnail before I delete, and fade in the next text on my stats boxes.

To do a fade you create a new Opacity animation

var fadeIn = new fx.Opacity($('StatusBoxes'));

Then you call its custom method giving it the starting and ending opacity.

fadeIn.custom(.1,1);

The same approach is used to shrink the thumbnail out, only this time will set some options.

var shrink = new fx.Height(images[3],{duration:1000,onComplete:function()
                                {
                                        $('recent').removeChild(images[3]);
                                        $('recent').insertBefore(newImg,images[0]);
                                }
                        });
shrink.custom(160,0);

I use the onComplete event handler to remove the element when im done shrinking it and too add the new one.

Thats all for now

We covered a lot of ground in this article, some old some new. If you have any suggestions, or requests for further explanation please leave a comment.

Webthumb is a small project of mine that takes a snapshot of a website and gives you thumbnails in 4 different sizes. Currently the site is rather boring and has a couple of usability problems especially on the pickup page. Over the next couple of days I'll be using various AJAX techniques to upgrade Webthumb, writing about the process as things happen.

When looking at a adding AJAX you have a couple decisions you'll want to make up front. One is what tools your going to use. In the webthumb case thats pretty easy. Webthumb is a simple PHP app and doesn't use a framework, so I need a nice general PHP/AJAX framework that is easy to use, HTML_AJAX fits that need. I may also need so animations or other effects, since im not planning on doing anything really fancy i'll use the extremly small but powerful moo.fx. I'll also keep scriptaculous in mind if I end up needing something fancier, but I think its overkill for this project.

After picking my tools I need to decide what my goals are. My main focus will be to improve usability, but I also want to use AJAX to make the site seem a bit flashier, so its a bit of a technology demo too.

Updating Thumbnail Pickup

The first page im targeting is the thumbnail pickup page. Thumbnail generation is an asyncronous process, you make a request which is added to a queue and the thumbnail processor walks the queue generating thumbnails updating the status when its done. The current way this is handle is by letting the user reload the page a bunch of times, but the speed of thumbnail generation isn't consistent so if things run slow its easy to thing something is broken. I'll be fixing this by using AJAX to poll the server on a 1 or 2 second interval, and use that information to make some sort of a progress bar.

HTML_AJAX and PHP remoting

HTML_AJAX supports a number of different ways to add AJAX to your pages, in this case I've chosen to use its PHP remoting supporting. This works registering a PHP class to be used by JavaScript, and then using a proxy object from JavaScript to make calls or making calls with the classname and methodname specified. Since im only going to be making a few calls im not going to be using a proxy object, if you want some examples on how that works checkout my Hello World with HTML_AJAX post.

The RequestStatus PHP class

The class i'll be exporting too JavaScript is call RequestStatus it currently has a single method, getStatus. If you were using it in PHP a call would look something like:

<?php

$status = new RequestStatus();
$jobStatus = $status->getStatus($jobId);

// job is in queue
if ($jobStatus['status'] == 'queue') {
  echo 'Job is in Queue, there are '.$jobStatus['wait'].' jobs in front of it';
}
else if ($jobStatus['status'] == 'processing') {
  echo 'Job is being processed, it started at'.$jobStatus['start'];
}
else if ($jobStatus['status'] == 'complete') {
  echo 'Job is complete';
}

?>

Exporting a PHP Class

The easiest way to make a PHP class available to JavaScript is too use the HTML_AJAX_Server class. If your using a framework you can either integrate this class into your front controller to handle AJAX dispatch or you can use the HTML_AJAX class directly and write your own equivalent of HTML_AJAX_Server to handle call dispatch. I run HTML_AJAX_Server inside of a file called ajax.php, an example of which is shown below. The only item of note is that im specify the methods on that class to export, doing this is always a good idea, and its the only way to keep method case in php4.

<?php

require_once 'lib/RequestStatus.class.php';
require_once 'HTML/AJAX/Server.php';

$server = new HTML_AJAX_Server();

$server->registerClass(new RequestStatus(),'RequestStatus',array('getStatus'));

$server->handleRequest();

?>

At this point a good thing is test that the class has been registered correctly and isn't thowing any PHP errors. To do this load the generated JavaScript stub up in your browser, ajax.php?stub=RequestStatus. For any class the result should look something like below:

// Client stub for the RequestStatus PHP Class
function RequestStatus(callback) {
	mode = 'sync';
	if (callback) { mode = 'async'; }
	this.className = 'RequestStatus';
	this.dispatcher = new HTML_AJAX_Dispatcher(this.className,mode,callback,'/webthumb/ajax.php?','JSON');
}
RequestStatus.prototype  = {
	Sync: function() { this.dispatcher.Sync(); },
	Async: function(callback) { this.dispatcher.Async(callback); },
	getStatus: function() { return this.dispatcher.doCall('getStatus',arguments); }
}

Making AJAX calls to RequestStatus

Now that the backend is setup its just a matter of making some AJAX calls on the thumbnail pickup page.

To do this we'll create a new JavaScript function will be run by the pages onload handler. It will start a timer that makes a call to getStatus every 2 seconds, as long as another request isn't currently being waited on. Will be using HTML_AJAX.call to make the requests.

To get the needed JavaScript libraries I specify a client parameter too ajax.php. Im grabbing all the default libraries and the optional alias library. Alias lets me type $('id) instead of document.getElementById.

<script type="text/javascript" src="ajax.php?client=all,alias"></script>

My status code check code is pretty simple, if the status is complete is reload the page, and let the PHP code display the images. If not I display a new status message, either the items ahead of this thumbnail job in the queue, or the time that processing started on the thumbnail. I also added a simple animated loading gif. The status check JavaScript is below, followed by the HTML it updates.

HTML_AJAX.defaultServerUrl = 'ajax.php';
var inProgress = false;
var jobId = '< ?php echo $jobId; ?>';
var statusInterval;

function setupStatusCheck() {
        statusInterval = window.setInterval(statusCheck,2000);
}
function statusCheck() {
        if (inProgress) {
                return;
        }

        inProgress = true;

        HTML_AJAX.call('RequestStatus','getStatus',statusCheckCallback,jobId);
}
function statusCheckCallback(result) {
        inProgress = false;

        if (result.status == 'complete') {
                window.clearInterval(statusInterval);
                window.location = '< ?php echo $_SERVER['PHP_SELF']; ?>?id='+jobId+'&reload=true';
        }
        else if (result.status == 'processing') {
                $('statusMessage').innerHTML = 'Thumbnail generation started at '+result.start;
        }
        else if (result.status == 'queue') {
                $('statusMessage').innerHTML = 'Your thumbnail is in the processing queue, there are '+result.wait+' items ahead of it';
        }
}

               <div id="statusBlock">
                        <p id="statusMessage">Your thumbnail is in the processing queue, there are < ?php echo $waiting; ?> items ahead of it.

This page will automatically update when the thumbnail is complete.

<img src="loading.gif"/>
                </div>

There are a number of other enhancements we could make to this page. First it could use some CSS cleanups to look nicer. You could also skip the last reload by adding the images using JavaScript. And finally I could make a nicer way to show the 4 thumbnails. These enhancements and some some fun project on the front page will be covered in future articles.

Oh and these changes are live, so you can check them out by visting webthumb and putting in a url.