There and Back Again

AJAX File upload Progress

Example of how to use the newest version is available in another post.
Update
There is a new version of this code that follows the same approach but uses HTML_AJAX instead of JPspan.
You can view the new demo, and view the code in websvn.

Also note that the server isn’t setup to accept files larger then 8meg, so anything bigger will fail.

Also I’m looking for someone to help me improve the error handling, if your interested in getting involved and making some regular releases of the new code let me know.

Original Post below

A couple days ago I found an interesting ruby on rails project. It uses AJAX to update a progress bar as the file uploads. The trick is a patch to rails for getting upload status and doing the upload in an iframe so that the main page is still active.

So to replicate this I just had to find a patch that provides upload status in PHP and then implment my little iframe upload widget.

I found the PHP with a little work from google: Upload Progress Meter

First you need to install the patch and the extension, the included instructions are easy to follow. The only problem I found is that: upload_progress_meter.store_method = “file” had to be set in my php.ini before thing would work.

I also ran into a JPSpan problem, if your having network problems the status call might take longer then 1 second, and you’ll get Call in Progres error alert. This can be fixed with the current version of JPSpan but i’d like to see some api added to help. The proxied objects need some type of inProgress call to make this an easy fix.

One item of note is that the extension only provides information in a way to provide 1 progress bar per form no matter how many files. The javascript code is setup so multiple forms could be on a page at once and both uploading, but this hasn’t been tested.

Here is the demo you’ve been waiting for, for most connections a 250k file will be enough to see something besides connecting and complete.

Also if anyone has the time and skills to review the php patch and see what it would take to get integrated, please let me know. I haven’t heard from its author so I don’t know why its not integrated but it just seems crazy to have a 3K patch that is this useful only available to those who are willing to patch.

Code walk through

So the basic flow happens like this.

Display a page with a form:
this page has a hidden iframe, a hidden progress div, and some extra javascript code

Select the file to upload and submit the form:
The form has a target of the hidden iframe so even though the throbber starts the main page view won’t be getting new content when the upload is done.

The form onclick handler fires a setup function:
The function finds the progress div and shows it, it also registers a function to update the status ever second.

The update function fires every second:
This functions checks a counter to see if there are any more divs to update, if the counter is 0 it stops the update function from firing again
The function creates a remote proxy object to the php class UploadProgressMeterStatus if it hasn’t already been created
The function calls the get_status method on the proxy object with a list of all the progress divs and UPLOAD_IDENTIFIER’s that we need status for.
The function exits

The get_status method on the php class is called
The method calls upload_progress_meter_get_info() for each passed in identifier, the information is formated to a percent and a message which is returned

The callback function for get_status is called when the PHP class returns data
The callback updates the progress div
If were at 100% the we decrement our progress div counter and remove us from the list of divs to be updated

The iframe can also load a page once the file upload is done, it doesn’t currently do anything.

Example Code

This is really the only interestin php code in the project, and its not that interesting. When you have the Upload Progress Meter extension installedany form that is doing a file upload and has a hidden var called UPLOAD_IDENTIFIER can be tracked. The identifier has to be passed into the upload_progress_meter_get_info function so you have to keep track of it on the javascript side. Here we just pass those ids in, do a bunch of formating on the resulting array, and return the results. Note the returned array isn’t documented anywhere so this code and the code in the example php code provided with the extension is the best place to start if you want to do something else with it.

       /**
         * Get the status of all uploads passed in
         */
        function get_status($ids) {
                $ret = array();
                foreach($ids as $id => $upId) {
                        $ret[$id] =  new stdClass();

                        $tmp = upload_progress_meter_get_info($upId);
                        if (!is_array($tmp)) {
                                $ret[$id]->message = "Complete";
                                $ret[$id]->percent = "100";
                                break;
                        }

                        if ($tmp['bytes_total'] < 1) {
                                $percent = 100;
                        }
                        else {
                                $percent = round($tmp['bytes_uploaded'] / $tmp['bytes_total'] * 100, 2);
                        }

                        if ($percent == 100) {
                                $ret[$id]->message = "Complete";
                        }

                        $eta            = sprintf("%02d:%02d", $tmp['est_sec'] / 60, $tmp['est_sec'] % 60 );
                        $speed          = $this->_formatBytes($tmp['speed_average']);
                        $current        = $this->_formatBytes($tmp['bytes_uploaded']);
                        $total          = $this->_formatBytes($tmp['bytes_total']);

                        $ret[$id]->message = "$eta left (at $speed/sec) $current/$total($percent%)";
                        $ret[$id]->percent = $percent;
                }
                return $ret;
        }

On the javascript side there is a bit more code, but none of its all that complex. This code is used to show the progress bar and give it some initial values. The main things to notice are that were add an update method to the div, this is a nice trick since it allows for runtime extension of objects in the DOM, and it will make updating things nice and easy in the other functions. Were also adding a getFirstDivByClass method to the div, I’m doing this so I don’t have to have so many divs to track, the classes only have to be unique inside the progress bar for this to work and thats much easier to achieve.

/**
 * Shows a progress bar and sets it to 0
 */
function UploadProgressMeter_EnableProgress(progress_id) {
        var progress = document.getElementById(progress_id);
        progress.style.display = 'block';
        progress.percent = 0;
        progress.message = "Connecting";

        progress.update = function() { this.getFirstDivByClass('bar').style.width = this.percent+'%'; this.getFirstDivByClass('message').innerHTML = this. message; }

        progress.getFirstDivByClass = function(className) {
                var nodes = this.getElementsByTagName('div');
                for(var i = 0; i < nodes.length; i++) {
                        if (nodes[i].className == className) {
                                return nodes[i];
                        }
                }

        }

        progress.update();
}

The code below calls the remote proxy and creates the callback function to handle the results. The is one area where improvements could be made. First there should be a check if there is currently a call in progress. Then it might also be smart to call the server less (especially on large files) and just generate the stats from the current download rate. This would add some complexity but would allow the progress bar to update smoothly and would allow the server calls to get down to once every 5 or 10 seconds.

If you don't do a lot of javscript programming its worth nothing the use of for(var prop in result) and delete UploadProgressMeter_active[prop];

for(var prop in result) is how you loop through the properties on an object, this allow you to use them as associative arrays (just watch for methods on the objects since you'll loop through them too).

delete UploadProgressMeter_active[prop] is the equivalent of unset($array['key']);


/**
 * Update the progress bars of all the current bars
 */
function UploadProgressMeter_Update() {
        if (UploadProgressMeter_count == 0) {
                clearInterval(UploadProgressMeter_intervalId);
                UploadProgressMeter_intervalId = false;
                return;
        }

        if (UploadProgressMeter_remote == false) {
                var callback = {
                        get_status: function(result) {
                                for(var prop in result) {
                                        if (prop != "toString") {
                                                document.getElementById(prop).percent = result[prop].percent;
                                                document.getElementById(prop).message = result[prop].message;
                                                document.getElementById(prop).update();

                                                if (document.getElementById(prop).percent == 100) {
                                                        UploadProgressMeter_count--;
                                                        delete UploadProgressMeter_active[prop];
                                                }
                                        }
                                }
                        }
                }
                UploadProgressMeter_remote = new uploadprogressmeterstatus(callback);
        }

        UploadProgressMeter_remote.get_status(UploadProgressMeter_active);
}

Code List

Updates

You may have noticed that the demo has stopped working a couple times. This was related to 2 things and there things you might want t think about if your going to use the patch. First it writes tmp files and fails silently if the directly no longer exists (darn /tmp cleaning scripts). Second its on my php5 server which has some users who are pushing the envelope which required me to upgrade to the php 5.1 beta and I forgot to repatch. Repatching wasn't a big deal though I did have to move where a function was declared to get the extension to compile in gcc 4 (I upgraded to fedora core 4 as well).

Anyhow the demo is working and it should stay working as long as I remember to repatch with each upgrade.

New Version

This code has been updated to work with HTML_AJAX and to handle error conditions better. I haven't done a formal release but you can grab it from svn.

Check it out from: http://svn.bluga.net/HTML_AJAX/UploadProgressMeter/trunk/

Or use websvn to get a tarball: http://websvn.bluga.net/wsvn/HTML_AJAX/UploadProgressMeter/trunk/?rev=0&sc=0

Also if your interested in helping out with the Upload Progress Meter let me know

249 thoughts on “AJAX File upload Progress

  1. Roy

    That is the example that I mentioned.
    When installed on my server, the popup window closed the moment I start uploading a file of decent size (2 MB).

    The demo on their and your servers worked really fine.
    My server is just trying to be difficult !! 🙁
    Im starting to believe in server gnomes.

  2. quentin

    i tried to apply the patch to php version 4.4.0, but failed.
    does the patch work with php version higher than 4.3.10? any solution?

  3. Joshua Eichorn Post author

    Quentin: You’ll have to wait until the patch author makes an update, i doubt the changes are big enough that it will take long, but you might try sending him an email.

  4. Roy

    Joshua, just wondering if I can pay you to install this mod.
    I really wanted to get it working but my server is as obstinate as a mule, wouldn’t even let me enjoy a cool mod just for once.
    Does it matter if it is a cPanel server? It shouldn’t right? Coz a server is a server, PHP is just PHP.

  5. Johnny

    Hi Josh,

    Is there anyway to implement the progress meter if you do not have access to the php.ini file (i.e. my site is hosted on an ISP which does not give me this freedom)

    Thanks!

  6. Joshua Eichorn Post author

    Johnny this solution is based on patching php to provide this functionality so its not really a possiblility unless you control the server. If you look around google there are some other projects that use a cgi or mod_perl to handle the status messages that might work for you.

  7. Gonzalo

    hello joshua!! how are you?

    i’ve trying to use your php program and I had a problem:

    when I try to upload some file its print an error:

    [Server_error] Only variables shold be assigned by reference while calling uploadprogressmeterstatus.get_status()

    you can help me?

    Regards,

  8. Joshua Eichorn Post author

    Well thats a php error, im not sure whats generating but I would try to make a test case for your code that can be run directly.

  9. Gonzalo

    good man I can explain what i’ve did, i downloaded your package, and unzip into a directory and tryied to upload, i dont know if I need to apply any patch to php and do other things, do you have any tutorial to install your package??? do you want to talk by msn messenger and the solution we post here?

    Regards!

  10. Gonzalo

    how I do it? i’ve tryied to apply the patch and the extension but I have php-4.4.0 compiled on linux debian and i’ve compiled the php with the patch and the extension, how do you did those things to make it works???

  11. Joshua Eichorn Post author

    Well I don’t see a version of the patch for 4.4.0, you can try the 4.3.10 patch im guessing it will still apply.

    If your used to building apache/php by yourself and applying patches then you might not want to go this route.

    The patch install instructions are included in the patch dir: http://pdoru.from.ro/upload-progress-meter/upload-progress-meter-v4.1/php-patch/INSTALL

    Then build your php install. After that you need to build the extension, the instructions are included again: http://pdoru.from.ro/upload-progress-meter/upload-progress-meter-v4.1/upload_progress_meter/INSTALL

    If you need more help then that, you’ll have to give me more details to what problems your having.

  12. Inshan Khairullah

    Hey Mr. Eichhorn, thanks for the demo. A php upload is exactly what I need. Unfortunately I am on a shared server (Dreamhost) and I don’t think they support extensions. I noticed on one of you side notes on (http://pdoru.from.ro/) about the “patch is no londer needed”. Murphically (law), the link or any info about this is not this. I am would like to inquire about this or if there is a solution that does not require such a patch. Your help is more than appreciated.

    Inshan Khairullah

  13. Joshua Eichorn Post author

    Well if you want a real progress bar (ie shows speed and % done) there aren’t a lot of options. You can either patch PHP and add in the extension or use another language that offers this support. I think mod_perl supports it out of the box, you should be able to find a package that helps with the php/mod_perl integration with a google search.

  14. Dave Koopman

    Hey, nice work, I am really impressed with the IFRAME implementation to provide the upload bar inline.

    I have my own version of this, started with the same source code. My improvement to the patch is: optional MySQL — instead of storing the upload data in a file, store it in a MySQL table. This is helpful when you have many web servers sitting behind a load balancer and cannot gaurantee that each request for progress will come into the same server. So, for the same reason that MySQL stored sessions are popular, I made this patch have the ability to store it’s upload progress data in a MySQL database table.

    Demo and Source Code available here:
    https://www.modphp.org/viewtopic.php?t=324

    I made a plug for you and your web log and gave link to your site.

    Keep up the good work! I doubt you remember meeting me, I met you at a AzPHP users group meeting many months ago, so when I was searching for PHP Upload Progress Bar and saw your name come up, it caught my attention.

  15. Joshua Eichorn Post author

    Dave:
    Thanks looks great, at some point I’m going to write another article about this, I’ll make sure to include your updated mysql backend, that looks a lot nicer then the file stuff. If your interested on colloborating on this let me know, my plan is to do just a basic im uploading effect for people without the extension (since you know when things are done) and show actual progress for people that have the extension setup.

    That way all those people out there on shared hosts or with no clue on howto compile PHP can still have something working.

  16. Dave Koopman

    Josh:
    Yes, I may be interested in colloborating in a new version… I think the patch works, the hard part is refining the look and feel using IFRAMES, JS, XML (AJAX). For users that cannot patch their source code, what do you think of coming up with some way to estimate the users bandwidth speed before the upload starts (using AJAX, of course!). Use AJAX to perform a bandwidth test, so that we make the “fake” progress bar an estimation rather than a pure guess. The problem is that we need to know the size of the upload before it starts also, I don’t think we can get that with Javascript (or can we)? Have you tried xajax? Just curious, I looked at it yesterday, I like how it modularizes the XML parsing, makes it easy for the common folk!

  17. Dave Koopman

    As you know, I am the author of the patch that extends the one you used. My version of this patch uses MySQL as the storage module for upload data, instead of files.

    You inspired me to do this: The patch doesn’t change, but after seeing your example and having a little time to chew on it, I wanted to develop a client of my patch that uses AJAX to display the progress bar. I also wanted to show the progress bar inline (instead of in a pop-up). So, after weeks of thinking about it, I finally did it. I am using XAJAX to constantly refresh the upload progress bar. I display the upload progress inline in an iframe, that refreshes it’s data through XAJAX. Thanks for the insight, this post was my inspiration to do it! I hope it helps somebody out.

    Source code and demo here: http://www.modphp.org/sutra664.php#664

  18. cj

    i had the problem with the upload status function always just returning bool(false) all the time, but the patch was installed right and i had followed all other instructions.

    after much hairpulling and bad words, i scrolled thru the entire php.ini file, and lo and behold, found ANOTHER upload related var outside the upload section of it.

    post_max_size. it was set to 8, and files larger than that are just ignored by php if you try to upload them.

    i changed this to 1000M (like the upload_max_filesize, i am building a site that will get large uploads), and the progress bar FINALLY starts working.

    so a tip for others with this problem:
    make sure your upload_max_filesize and post_max_size are set right.

  19. Ranjan Gupta

    I am facing the following problem in Opera running
    on Fedora Core 3 Linux

    if you go to musicindiaonline.com and try to play any song
    the following error comes:

    Javascript Error: Methods missing!

    Please suggest a way around so that I can play the songs.

  20. Naikel Aparicio

    I installed the PHP patch in PHP 5.0.5 and recompiled it and I had several problems with your demo. Most of the problems were because the INSTALL and README files said the upload meter was called upload_progress_tracking and it is upload_progress_meter.

    So, I run into the problem of the bar going to “Complete” seconds after starting the upload. I noticed the tmp directory was created ok, permissions were fine and everything. The patch showed in phpinfo, everything enabled, but nothing worked. Why? because my settings in the httpd.conf were “upload_progress_tracking*” and the extension setting in the php.ini was “upload_progress.tracking.so” instead of “upload_progress_meter*”.

    Now everything works fine, after about 2 hours of wasted time 😛

  21. Naikel Aparicio

    BTW, for the “[Client_error] Call in progress” problem, a quick solution is to add these following line to the UploadProgressMeter.js:

    Just below the line:
    UploadProgressMeter_remote = new uploadprogressmeterstatus(callback);

    Add these line:
    UploadProgressMeter_remote.clientErrorFunc = function(e) { return false; }

    Dirty solution, but it works.

  22. Naikel Aparicio

    After working a while with this upload progress meter, which works pretty well I must say, I have trouble detecting incomplete files.

    How could you detect an incomplete upload?

    If I press “stop” while uploading a file in the browser (forcing an incomplete upload):

    1.- The upload progress meter says “Complete”.
    1.- The form “action URL” is called (why?).
    2.- is_uploaded_file($_FILES[‘upload’][‘tmp_name’]) == true like if the file was complete.
    3.- $FILES[‘upload’][‘error’] == UPLOAD_ERR_OK like if there were no problems at all.

    Is there a way to detect the incomplete file, so the “URL form action” is not called and the upload progress meter says “Incomplete” ?!? (can’t really figure it out).

  23. Joshua Eichorn Post author

    Naikel:

    First of all the form action has to be called thats how the file is uploaded. File’s can’t be submitted using XMLHttpRequest, they have to be submitted using a form. You just use a hidden iframe so the main page doesn’t reload.

    As far as status goes and detecting errors check out the updated version that uses HTML_AJAX
    http://websvn.bluga.net/wsvn/HTML_AJAX/UploadProgressMeter/trunk/?rev=0&sc=0

    You can grab a tarball download right from the svn viewer. Or if you want to check it out just goto:
    http://svn.bluga.net/HTML_AJAX/UploadProgressMeter/trunk/

    You can install HTML_AJAX using the pear installer

  24. Patrick

    I’ve got the patch installed, but i’m getting this error, which is the same as the error i was getting before the patch.
    On PHP 4.4.1 Apache 2

    -[server_error] only server variables should be assigned by reference while calling uploadprogressmeterestatus.get_status()

    While these messages are very useful, I wish it gave any sort of idea to where the error born, so I could attemp to locate it.

  25. Joshua Eichorn Post author

    Thats a reference warning, stupid php 4.4

    Are you trying the new version that uses HTML_AJAX with the newest version of HTML_AJAX

  26. Gordito

    AJAX is not a framework. It’s an acronym for Asynchronous JavaScript and XML.

  27. laurent

    Hello,
    I’m using Apache 2 with PHP 5.0.4 over Windows XP.
    I untar/unziped the package UploadProgressMeter.tar.gz into a directory under Apache. Tested the file demo.php in the archive : browsed for a file to upload and click the ‘Upload File’ button. I’m getting the following message box error :
    [Server_Error] Syntax error while calling uploadprogressmetterstatus.get_status()
    How can I fix that problem ? Thank you

  28. Lz

    I have sample up and working, but not very good with the javascript and iframes. I know php very well though. My question is where do I put the code to handle the uploaded file? The form is never offically posted to action page in form tag so code never gets executed. I also tried to modify the class and add function called processfile() which was triggered at 100%. The problem was the $_FILE variables where never set. I know this is probably simple question for most, but having little trouble figuring out where to put process the file code.

  29. Joshua Eichorn Post author

    The file is uploaded to the action, it just happens in the iframe so you don’t see the results. For testing you might want to update the iframe code so that its visible instead of hidden.

  30. Lz

    Joshua thanks for the help. I figured out how to enlarge the iframe so could see the output and debug. It was just simple logic error on my part.

  31. Himanshu

    Hi!

    Its really a good application. I’ve downloaded the original one as well as the newer version and made changes to the older one. Its working fine, only the blue progress bar; but the problem is with the data summary below the bar. It is not showing me the summary. I’ve tried my best to get what the problem is but I cann’t. Please tell me what to do as it is needed urgently.

    one more thing, when I’m loading my page it is giveing me JS error: ‘HTML_AJAX’ is undefined. I’ve also copied the HTML_AJAX folder to the main folder and configured it but this error is not solving.

    Many Thanks,
    Himanshu

  32. Himanshu

    Again, I’ve not installed any patch on my system, just Apache-1.3.14 & PHP-5.1.1 and then copied the upload folder to apache folder. Is there any need to install the patch first before this application or it’ll work fine without installing? If yes, please let me know the path where I can find the patch as well as the instruction to install and complile. I’m working on windows NT system.

    Waiting..
    Himanshu

  33. Joshua Eichorn Post author

    The newest version should work without the patch but you can’t get progress information without it.

    You should be able to update it too provide some kind of im uploading icon and then a done message.

  34. Pingback: ePHP.pl » Notki » Pasek postępu przy uploadzie plików

  35. Adam

    apache 2.0.54 PHP 5.0.4 LibGD 2.0.33 – Mac os 10.4
    After making sure i compiled using the php 5 phpize and I had to add static int check_identifier(char * identifier) to be declared at the start of the C file, I now get the following error on starting apache
    dyld: lazy Symbol binding failed: Symbol not found: _upload_progress_register_callback
    referenced from :/usr/local/phpexten/upload_progress_meter.so
    expecred in :flat namespace

    any clues as to what this is now complaining about ?

  36. Patrick

    In the new firefox 1.5 (deerpark) the progress meter dies around 95%. This is not happening in IE 6.

    My guess is that the /var/uploadbar/upl_{$upId}.txt no longer exists.

    I tried these two bits of code in the ID loop of the getStatus method of the UploadProgressMeterStatus:

    if(!chmod(“/var/uploadbar/upl_{$upId}.txt”,0777)){
    $ret[$id]->message = “File Error”;
    $ret[$id]->percent = “Probably Done”;
    break;
    }

    if(!file_exists(“/var/uploadbar/upl_{$upId}.txt”)){
    $ret[$id]->message = “Complete”;
    $ret[$id]->percent = “100”;
    break;
    }

    neither of them worked. Also, I had to add @chmod(“/var/uploadbar/upl_{$upId}.txt”,0777) in the top of the loop, to set the file readable. Do you know how I can fix the php setting to make this file readable to the upload_progress_meter_get_info() extension?

    I’m guessing that is the problem. Any thought on why FF is killing the script but IE finishes it?

    -Pat

  37. Joshua Eichorn Post author

    Well that file is created by the file upload extension so I don’t see why browser differences would affect it in any way. Also once the upload is complete that file is deleted so looking for it isn’t a good debugging method unless your doing uploads that take a large amount of time.

    Permission wise just make sure that PHP can create files in /var/uploadbar, they will automatically get created with the right permissions.

    But again, if it works in IE but not in FF then its not a problem with the extension, its with some other part of the code.

  38. Patrick

    I stand corrected, I did not need the @chmod in the getStatus method, but for some reason in FF 1.5 on my machine I’m getting the following at 95% for all uploads

    In a FF popup:
    A script on this page may be busy, or it may have stopped responding. You can stop the script now, or you can continue to see if the script will complete. [kill script][continue]

    If I kill script, it will move to complete. If i continue script, the page stalls permanently. I went back to FF 1.02 and no problems, and no problems in IE. I wonder if this has something todo with the new FF.

  39. Joshua Eichorn Post author

    Something in the new FF is triggering a bug. It sounds like your ending up in an infinite loop or something. Are you using the new version of the upload code from SVN or the older code.

    I haven’t seen that problem using FF 1.5 running against the demo at: http://php5.bluga.net/UploadProgressMeter/demo.php. But thats the new version of the code that uses HTML_AJAX not the older version that used JPSpan.

  40. Patrick

    It must be some unique combination of my internet connection and browser. I’ve test FF 1.5 on another machine (outside my network) and was able to use the script just fine. However, my setup is also failing on your demo site. Yes, I’m using the most recent version.

    Same symptoms on your demo. Work in IE, not in FF 1.5

    Weird