There and Back Again

AJAX File upload Progress

Example of how to use the newest version is available in another post.
Update
There is a new version of this code that follows the same approach but uses HTML_AJAX instead of JPspan.
You can view the new demo, and view the code in websvn.

Also note that the server isn’t setup to accept files larger then 8meg, so anything bigger will fail.

Also I’m looking for someone to help me improve the error handling, if your interested in getting involved and making some regular releases of the new code let me know.

Original Post below

A couple days ago I found an interesting ruby on rails project. It uses AJAX to update a progress bar as the file uploads. The trick is a patch to rails for getting upload status and doing the upload in an iframe so that the main page is still active.

So to replicate this I just had to find a patch that provides upload status in PHP and then implment my little iframe upload widget.

I found the PHP with a little work from google: Upload Progress Meter

First you need to install the patch and the extension, the included instructions are easy to follow. The only problem I found is that: upload_progress_meter.store_method = “file” had to be set in my php.ini before thing would work.

I also ran into a JPSpan problem, if your having network problems the status call might take longer then 1 second, and you’ll get Call in Progres error alert. This can be fixed with the current version of JPSpan but i’d like to see some api added to help. The proxied objects need some type of inProgress call to make this an easy fix.

One item of note is that the extension only provides information in a way to provide 1 progress bar per form no matter how many files. The javascript code is setup so multiple forms could be on a page at once and both uploading, but this hasn’t been tested.

Here is the demo you’ve been waiting for, for most connections a 250k file will be enough to see something besides connecting and complete.

Also if anyone has the time and skills to review the php patch and see what it would take to get integrated, please let me know. I haven’t heard from its author so I don’t know why its not integrated but it just seems crazy to have a 3K patch that is this useful only available to those who are willing to patch.

Code walk through

So the basic flow happens like this.

Display a page with a form:
this page has a hidden iframe, a hidden progress div, and some extra javascript code

Select the file to upload and submit the form:
The form has a target of the hidden iframe so even though the throbber starts the main page view won’t be getting new content when the upload is done.

The form onclick handler fires a setup function:
The function finds the progress div and shows it, it also registers a function to update the status ever second.

The update function fires every second:
This functions checks a counter to see if there are any more divs to update, if the counter is 0 it stops the update function from firing again
The function creates a remote proxy object to the php class UploadProgressMeterStatus if it hasn’t already been created
The function calls the get_status method on the proxy object with a list of all the progress divs and UPLOAD_IDENTIFIER’s that we need status for.
The function exits

The get_status method on the php class is called
The method calls upload_progress_meter_get_info() for each passed in identifier, the information is formated to a percent and a message which is returned

The callback function for get_status is called when the PHP class returns data
The callback updates the progress div
If were at 100% the we decrement our progress div counter and remove us from the list of divs to be updated

The iframe can also load a page once the file upload is done, it doesn’t currently do anything.

Example Code

This is really the only interestin php code in the project, and its not that interesting. When you have the Upload Progress Meter extension installedany form that is doing a file upload and has a hidden var called UPLOAD_IDENTIFIER can be tracked. The identifier has to be passed into the upload_progress_meter_get_info function so you have to keep track of it on the javascript side. Here we just pass those ids in, do a bunch of formating on the resulting array, and return the results. Note the returned array isn’t documented anywhere so this code and the code in the example php code provided with the extension is the best place to start if you want to do something else with it.

       /**
         * Get the status of all uploads passed in
         */
        function get_status($ids) {
                $ret = array();
                foreach($ids as $id => $upId) {
                        $ret[$id] =  new stdClass();

                        $tmp = upload_progress_meter_get_info($upId);
                        if (!is_array($tmp)) {
                                $ret[$id]->message = "Complete";
                                $ret[$id]->percent = "100";
                                break;
                        }

                        if ($tmp['bytes_total'] < 1) {
                                $percent = 100;
                        }
                        else {
                                $percent = round($tmp['bytes_uploaded'] / $tmp['bytes_total'] * 100, 2);
                        }

                        if ($percent == 100) {
                                $ret[$id]->message = "Complete";
                        }

                        $eta            = sprintf("%02d:%02d", $tmp['est_sec'] / 60, $tmp['est_sec'] % 60 );
                        $speed          = $this->_formatBytes($tmp['speed_average']);
                        $current        = $this->_formatBytes($tmp['bytes_uploaded']);
                        $total          = $this->_formatBytes($tmp['bytes_total']);

                        $ret[$id]->message = "$eta left (at $speed/sec) $current/$total($percent%)";
                        $ret[$id]->percent = $percent;
                }
                return $ret;
        }

On the javascript side there is a bit more code, but none of its all that complex. This code is used to show the progress bar and give it some initial values. The main things to notice are that were add an update method to the div, this is a nice trick since it allows for runtime extension of objects in the DOM, and it will make updating things nice and easy in the other functions. Were also adding a getFirstDivByClass method to the div, I’m doing this so I don’t have to have so many divs to track, the classes only have to be unique inside the progress bar for this to work and thats much easier to achieve.

/**
 * Shows a progress bar and sets it to 0
 */
function UploadProgressMeter_EnableProgress(progress_id) {
        var progress = document.getElementById(progress_id);
        progress.style.display = 'block';
        progress.percent = 0;
        progress.message = "Connecting";

        progress.update = function() { this.getFirstDivByClass('bar').style.width = this.percent+'%'; this.getFirstDivByClass('message').innerHTML = this. message; }

        progress.getFirstDivByClass = function(className) {
                var nodes = this.getElementsByTagName('div');
                for(var i = 0; i < nodes.length; i++) {
                        if (nodes[i].className == className) {
                                return nodes[i];
                        }
                }

        }

        progress.update();
}

The code below calls the remote proxy and creates the callback function to handle the results. The is one area where improvements could be made. First there should be a check if there is currently a call in progress. Then it might also be smart to call the server less (especially on large files) and just generate the stats from the current download rate. This would add some complexity but would allow the progress bar to update smoothly and would allow the server calls to get down to once every 5 or 10 seconds.

If you don't do a lot of javscript programming its worth nothing the use of for(var prop in result) and delete UploadProgressMeter_active[prop];

for(var prop in result) is how you loop through the properties on an object, this allow you to use them as associative arrays (just watch for methods on the objects since you'll loop through them too).

delete UploadProgressMeter_active[prop] is the equivalent of unset($array['key']);


/**
 * Update the progress bars of all the current bars
 */
function UploadProgressMeter_Update() {
        if (UploadProgressMeter_count == 0) {
                clearInterval(UploadProgressMeter_intervalId);
                UploadProgressMeter_intervalId = false;
                return;
        }

        if (UploadProgressMeter_remote == false) {
                var callback = {
                        get_status: function(result) {
                                for(var prop in result) {
                                        if (prop != "toString") {
                                                document.getElementById(prop).percent = result[prop].percent;
                                                document.getElementById(prop).message = result[prop].message;
                                                document.getElementById(prop).update();

                                                if (document.getElementById(prop).percent == 100) {
                                                        UploadProgressMeter_count--;
                                                        delete UploadProgressMeter_active[prop];
                                                }
                                        }
                                }
                        }
                }
                UploadProgressMeter_remote = new uploadprogressmeterstatus(callback);
        }

        UploadProgressMeter_remote.get_status(UploadProgressMeter_active);
}

Code List

Updates

You may have noticed that the demo has stopped working a couple times. This was related to 2 things and there things you might want t think about if your going to use the patch. First it writes tmp files and fails silently if the directly no longer exists (darn /tmp cleaning scripts). Second its on my php5 server which has some users who are pushing the envelope which required me to upgrade to the php 5.1 beta and I forgot to repatch. Repatching wasn't a big deal though I did have to move where a function was declared to get the extension to compile in gcc 4 (I upgraded to fedora core 4 as well).

Anyhow the demo is working and it should stay working as long as I remember to repatch with each upgrade.

New Version

This code has been updated to work with HTML_AJAX and to handle error conditions better. I haven't done a formal release but you can grab it from svn.

Check it out from: http://svn.bluga.net/HTML_AJAX/UploadProgressMeter/trunk/

Or use websvn to get a tarball: http://websvn.bluga.net/wsvn/HTML_AJAX/UploadProgressMeter/trunk/?rev=0&sc=0

Also if your interested in helping out with the Upload Progress Meter let me know

249 thoughts on “AJAX File upload Progress

  1. Pingback: Full(o)bloG » Blog Archive » varie

  2. Peter Nagy

    Hi Joshua!

    It’s really interesting to see, how a paradigm change(AJAX) changes the requirements of a serverside scripting language (Upload Progress Meter patch). I think one need to re-think the limits between “serverside” and “clientside”… ๐Ÿ™‚
    I wish if there were such progress patches to background SQL or socket processes… I think this is the only thing left to seamlessly integrate serverside processing into the client GUI.

    NP

  3. Christopher Thompson

    I have installed the upload patch and got it to work as well. I read that the author had submitted it to the PHP Group, but that there were both problems getting it in 5.0.x releases (too much else to do) and that it was not thread safe as I recall. Not sure what the status is now.

    I like the Ajax idea.

  4. Emil Tamas

    Hello, I reached to the same solution, using that patch. Sadly, sometimes is hard to patch PHP (eg. no shell access, wierd php builds like Ensim Webpliance, no admin rights and other stuff).
    Another possible solution to this would be to use Apache Hooks (similar to PERL’s Apache::UploadProgress, see more at this link http://www.omniti.com/~george/talks/Apache_Hooks-PHPCON2002.html) but the progress for this feature of PHP seems to be stalled).
    Another would be to use the hook features available in PERL, calling PERL objects from PHP using this PECL extension: http://pecl.php.net/package/perl

  5. Emil Tamas

    Also, please stop using this ugly term AJAX, we don’t need another term for JavaScript RPC, AJAX is a toilet cleaner or a soccer team and a hype that I hope will vanish soon.

  6. Joshua Eichorn Post author

    Like I said before, I’m not a big fan of the term, but its the term. Me uses it or not using isn’t going to change that. So since I can’t affect its usage im going to use the term that helps people know what im talking about and help people find my writings in google et al.

  7. Joshua Eichorn Post author

    Emil: I don’t really see non default extensions as the solution either, I just don’t see Apache hooks ever getting stable and though the perl trick is interesting I don’t see many people (including me) wanting to use a hack like that to do. Hopefully I can find someone to fix the patch and get it integrated.

  8. ali khalili

    Hello , I have started programming with php in AJAX framework.
    I want to know that is there any AJAX tools for PHP 5,something like JPSPAN that maps php5 classes to javascript world.
    Thanks for your good weblog ๐Ÿ™‚

  9. Joshua Eichorn Post author

    JPSpan works just fine in PHP5, i have a project using it actually. It won’t run under E_STRICT but thats not a big deal since JPSpan code only gets run in the server script so you can easily special case its error handling.

  10. Sriram

    Hello,

    This looks really great. But i had a problem when uploading the file. I get a javascript error saying [Client_Error][404] Microsoft-IIS/5.1 while calling uploadprogressmeterstatus.get_status(). I just need to know where will be this file uploaded?

    Thanks & Regards,
    Sriram.K

  11. John

    I had a problem when uploading the file. I get a javascript error saying [Client_Error][404] syntax error while calling uploadprogressmeterstatus.get_status().

  12. Francesco

    local error:

    [Client error] [404] microsoft IIS/5.1 while calling uploadprogressMeterStatus.get_status()

  13. Joshua Eichorn Post author

    Are these errors trying the demo my server or locally. If it locally have you tried the demo that comes with the patch to make sure its installed and setup correctly.

  14. Adrian Cid

    In my server don’t work I have php 5.0.2, and I have the same error. Please somebody help my.

  15. Philip Ashlock

    As I demo this in firefox 1.04 and IE 6 I see the bar instantly jump to Completed just as the file begins to upload. This seems to be the case with files of any size. Why might this be happening?

  16. dms

    Hi Joshua,
    I tested your Demo on your Server and it only works in Internet Explorer(Win) for me.

    Firefox at Linux shows me this Error: [Client_Error] Call in progress
    And a litle later this one: [Client_Error] Operation timed out: get_status while calling uploadprogressmeterstatus.get_status()

    Opera at Win creates this Error: [Client_Error] Unable to create XMLHttpRequest

    And Opera at Linux this one:
    [TypeError] Statement on line 601: object required
    Backtrace:
    Line 601 of linked script http://php5.bluga.net/progressDemo/demoserver.php?client
    this.http.setRequestHeader(“Content-Length”, this.body.length);
    Line 304 of linked script http://php5.bluga.net/progressDemo/demoserver.php?client
    request.prepare(this.xmlhttp);
    Line 897 of linked script http://php5.bluga.net/progressDemo/demoserver.php?client
    this.__client.asyncCall(request, this.__responseHandler, callName);
    Line 885 of linked script http://php5.bluga.net/progressDemo/demoserver.php?client
    return this.__asyncCall(this.__request, callName);
    Line 961 of linked script http://php5.bluga.net/progressDemo/demoserver.php?client
    return this.__call(url, arguments, “get_status”);
    Line 136 of linked script http://php5.bluga.net/progressDemo/UploadProgressMeter.js
    UploadProgressMeter_remote.get_status(UploadProgressMeter_active);
    At unknown location
    [statement source code not available]

  17. Joshua Eichorn Post author

    I’m not surprised by errors with Opera, i don’t think it has close to a working xmlhttprequest client until you get the newest version. Anyhow workaround for those problems would be patches to JPSpan. The timeout you got on firefox on linux sounds a little odd, that means the call didn’t complete within 3 seconds i think. Is that machine on a slow internet connection?

  18. N8mare

    Hello,
    i have a problem applying the patch. I use xampp and WinXP. Can anyone help me?
    Of course i get this error without the patch:
    [ClientError][404] syntax error while calling uploadprogressmeterstatus.getstatus().
    but the install instructions are only for unix systems…

  19. Pingback: TutorMachine » Blog Archive » AJAX - PHP mit Javascript mit XML

  20. Daniel

    Hi Joshua,
    Could you help me out? The progress bar keeps jumping from 0 to 100 instantly and I don’t quite understand how to save the file to a location on the server.
    Could you help me out?

    Thanks for this project.
    Daniel

  21. Joshua Eichorn Post author

    If the bar jump from 0 to 100 either your testing with too small of a file for your connection (could be quite big testing on localhost) or the patch and extension aren’t working right. Check and make sure a file is created in /tmp while an upload is taking place.

    To save the file to a location on the server use move_uploaded_file just like you would without the progress monitoring.

  22. Daniel

    hey Josh,

    I got it to work! Thanx a million. Turned out testing on localhost was just too fast for the progressbar to get going ๐Ÿ™‚

    Then I found this program called Charles that can throttle your bandwidth to any speed you choose (it’s great for debugging purposes). And there was my upload-progressbar ๐Ÿ™‚

    Bless ya. Daniรซl

  23. Daniel

    BTW, is there anything I can do about the “[Client_error] Call in progress”? It pops up every now and then… I suppose I could just suppress the error by not calling alert, but is there some way to fix the problem more thoroughly?

  24. ceejayoz

    Just tried 500kb, 1.4mb, 5.9mb, and 300mb files – all said “complete” within seconds on your demo page.

    Something’s not quite working, methinks, unless my ISP drastically upgraded my upload speed… :p

    Firefox 1.0.4 on a Mac Mini.

  25. Joshua Eichorn Post author

    Ok so here is a neat trick, the extension writes to a tmp file, i had it set to use a directory in /tmp for its storage. Every month some script was killing everything in /tmp. I moved the tmp file to a place that won’t get destoryed.

    The demo is working again.

  26. Jesse

    Where are the files that I choose to upload actually uploaded to? Or is the code to upload the file missing? I can’t seem to find any move codes.

  27. Joshua Eichorn Post author

    Jesse: to the normal php file upload location, im just ignoring them in the demo. The page your posting the form too would handle the file using move_uploaded_file just like normal.

  28. Noel da Costa

    Hi Joshua,

    I tried your demo (on your site) – Mac Safari and Firefox give this error:
    [ClientError] Operation timed out: getstatus while calling uploadprogressmeterstatus.getstatus()

    This is on a 2MB ADSL connection, uploading a relatively small file.

  29. Joshua Eichorn Post author

    The demo is broken again due to an upgrade to php 5.1, thats the problem with patches, you have to remember to apply them at every upgrade.

  30. Luke Douglas

    Is there a way to use this on multiple files? I have a routine that the user can indicate the number of files to be uploaded and then is presented with the number of Browse inputs that they want. I would like to show the progress of each file as it is being uploaded.

    Any ideas?

  31. Unnati Sethi

    I have a similar requirement to Lukes (display progress of multiple files download) and I am using AJAX. I have a web application module where the user can indicate the files to be downloaded and while the files are retrieved from a remote server I would like to show the status of each file as it is being retrieved (completed/failed).

    I am currently doing this in the same way mentioned by you above but my problem is that the minute the hidden forms response object returns the XMLHHTPRequest loops ends ie it doesnt continue trying to find out if all the files have been retieved. I cannot understand why. shouldnt two different frames submit be independent of each other?

    Please respons asap. I need this to work!

  32. Joshua Eichorn Post author

    Unnati: I haven’t tried using multiple iframes to solve the problem, but that seems like it should work.

    Without seeing your code I can’t think of any tips, feel free to email me or post a link to the source and i’ll take a look.

  33. +anindya

    Sorry guys, am trying to track a friend, googled and got just this page!

    and to her:
    OOse, is that you? respond asap, I’ve been trying to contact you for long! the ID is anindya[at]siggraph[dot]org. cheers!

  34. Joshua Eichorn Post author

    I would guess the crash is AJAX related, but im not sure. The two things I can think of that might be causing problems are something in the code is leaking javascript objects, or Safari doesn’t like using an iform as a form target.

  35. Roy

    Joshue,

    Interesting hack youve go there!

    I tried to implement it in vain.
    I noticed there is a test.php.
    What if the output is bool(false)?
    Does it mean that I didnt install the patch the right way?
    And when I go to /tmp/uploadbar/ is didnt see any text file. there shold be one created as a file is being uploaded…… ๐Ÿ™

    At this point, its still jumping from 0-100.

    Cant get it to work for the life of me.

  36. Joshua Eichorn Post author

    Roy,

    It sounds like your having problems with the patch, if you got it installed right you should see it in your phpinfo screen. I would also test things with the example code from the patch site since its simpler since there is no AJAX involved.

  37. Roy

    Hey Josh,

    I think my PHP was semi patched.

    On my PHPINFO, I see:

    Upload Progress Meter

    upload_progress_meter support : enabled
    available backend modules : file

    I tried the example code but the windows closed on me shortly after it popped up.
    Im setting up an image hosting thingy so this thing is going to impress the uploader, when I get it working that is. ๐Ÿ˜€