There and Back Again

AJAX File upload Progress

Example of how to use the newest version is available in another post.
Update
There is a new version of this code that follows the same approach but uses HTML_AJAX instead of JPspan.
You can view the new demo, and view the code in websvn.

Also note that the server isn’t setup to accept files larger then 8meg, so anything bigger will fail.

Also I’m looking for someone to help me improve the error handling, if your interested in getting involved and making some regular releases of the new code let me know.

Original Post below

A couple days ago I found an interesting ruby on rails project. It uses AJAX to update a progress bar as the file uploads. The trick is a patch to rails for getting upload status and doing the upload in an iframe so that the main page is still active.

So to replicate this I just had to find a patch that provides upload status in PHP and then implment my little iframe upload widget.

I found the PHP with a little work from google: Upload Progress Meter

First you need to install the patch and the extension, the included instructions are easy to follow. The only problem I found is that: upload_progress_meter.store_method = “file” had to be set in my php.ini before thing would work.

I also ran into a JPSpan problem, if your having network problems the status call might take longer then 1 second, and you’ll get Call in Progres error alert. This can be fixed with the current version of JPSpan but i’d like to see some api added to help. The proxied objects need some type of inProgress call to make this an easy fix.

One item of note is that the extension only provides information in a way to provide 1 progress bar per form no matter how many files. The javascript code is setup so multiple forms could be on a page at once and both uploading, but this hasn’t been tested.

Here is the demo you’ve been waiting for, for most connections a 250k file will be enough to see something besides connecting and complete.

Also if anyone has the time and skills to review the php patch and see what it would take to get integrated, please let me know. I haven’t heard from its author so I don’t know why its not integrated but it just seems crazy to have a 3K patch that is this useful only available to those who are willing to patch.

Code walk through

So the basic flow happens like this.

Display a page with a form:
this page has a hidden iframe, a hidden progress div, and some extra javascript code

Select the file to upload and submit the form:
The form has a target of the hidden iframe so even though the throbber starts the main page view won’t be getting new content when the upload is done.

The form onclick handler fires a setup function:
The function finds the progress div and shows it, it also registers a function to update the status ever second.

The update function fires every second:
This functions checks a counter to see if there are any more divs to update, if the counter is 0 it stops the update function from firing again
The function creates a remote proxy object to the php class UploadProgressMeterStatus if it hasn’t already been created
The function calls the get_status method on the proxy object with a list of all the progress divs and UPLOAD_IDENTIFIER’s that we need status for.
The function exits

The get_status method on the php class is called
The method calls upload_progress_meter_get_info() for each passed in identifier, the information is formated to a percent and a message which is returned

The callback function for get_status is called when the PHP class returns data
The callback updates the progress div
If were at 100% the we decrement our progress div counter and remove us from the list of divs to be updated

The iframe can also load a page once the file upload is done, it doesn’t currently do anything.

Example Code

This is really the only interestin php code in the project, and its not that interesting. When you have the Upload Progress Meter extension installedany form that is doing a file upload and has a hidden var called UPLOAD_IDENTIFIER can be tracked. The identifier has to be passed into the upload_progress_meter_get_info function so you have to keep track of it on the javascript side. Here we just pass those ids in, do a bunch of formating on the resulting array, and return the results. Note the returned array isn’t documented anywhere so this code and the code in the example php code provided with the extension is the best place to start if you want to do something else with it.

<?php

       /**
         * Get the status of all uploads passed in
         */
        function get_status($ids) {
                $ret = array();
                foreach($ids as $id => $upId) {
                        $ret[$id] =  new stdClass();

                        $tmp = upload_progress_meter_get_info($upId);
                        if (!is_array($tmp)) {
                                $ret[$id]->message = "Complete";
                                $ret[$id]->percent = "100";
                                break;
                        }

                        if ($tmp['bytes_total'] < 1) {
                                $percent = 100;
                        }
                        else {
                                $percent = round($tmp['bytes_uploaded'] / $tmp['bytes_total'] * 100, 2);
                        }

                        if ($percent == 100) {
                                $ret[$id]->message = "Complete";
                        }

                        $eta            = sprintf("%02d:%02d", $tmp['est_sec'] / 60, $tmp['est_sec'] % 60 );
                        $speed          = $this->_formatBytes($tmp['speed_average']);
                        $current        = $this->_formatBytes($tmp['bytes_uploaded']);
                        $total          = $this->_formatBytes($tmp['bytes_total']);

                        $ret[$id]->message = "$eta left (at $speed/sec) $current/$total($percent%)";
                        $ret[$id]->percent = $percent;
                }
                return $ret;
        }

?>

On the javascript side there is a bit more code, but none of its all that complex. This code is used to show the progress bar and give it some initial values. The main things to notice are that were add an update method to the div, this is a nice trick since it allows for runtime extension of objects in the DOM, and it will make updating things nice and easy in the other functions. Were also adding a getFirstDivByClass method to the div, I’m doing this so I don’t have to have so many divs to track, the classes only have to be unique inside the progress bar for this to work and thats much easier to achieve.

The code below calls the remote proxy and creates the callback function to handle the results. The is one area where improvements could be made. First there should be a check if there is currently a call in progress. Then it might also be smart to call the server less (especially on large files) and just generate the stats from the current download rate. This would add some complexity but would allow the progress bar to update smoothly and would allow the server calls to get down to once every 5 or 10 seconds.

If you don’t do a lot of javscript programming its worth nothing the use of for(var prop in result) and delete UploadProgressMeter_active[prop];

for(var prop in result) is how you loop through the properties on an object, this allow you to use them as associative arrays (just watch for methods on the objects since you’ll loop through them too).

delete UploadProgressMeter_active[prop] is the equivalent of unset($array['key']);


Code List

Updates

You may have noticed that the demo has stopped working a couple times. This was related to 2 things and there things you might want t think about if your going to use the patch. First it writes tmp files and fails silently if the directly no longer exists (darn /tmp cleaning scripts). Second its on my php5 server which has some users who are pushing the envelope which required me to upgrade to the php 5.1 beta and I forgot to repatch. Repatching wasn’t a big deal though I did have to move where a function was declared to get the extension to compile in gcc 4 (I upgraded to fedora core 4 as well).

Anyhow the demo is working and it should stay working as long as I remember to repatch with each upgrade.

New Version

This code has been updated to work with HTML_AJAX and to handle error conditions better. I haven’t done a formal release but you can grab it from svn.

Check it out from: http://svn.bluga.net/HTML_AJAX/UploadProgressMeter/trunk/

Or use websvn to get a tarball: http://websvn.bluga.net/wsvn/HTML_AJAX/UploadProgressMeter/trunk/?rev=0&sc=0

Also if your interested in helping out with the Upload Progress Meter let me know

249 thoughts on “AJAX File upload Progress

  1. Joshua Eichorn Post author

    Kris: you need to compile the extension, pecl install uploadprogress will do it

    If your on a hosted setup you’ll need your hosting company to support the extension

    If thats the case, there is support in apc extension for tracking things (http://us.php.net/apc) and that might be enabled already. You’ll just have to update the code to use APC’s rfc1867 stuff

  2. Ichibod

    Hey Josh… thanks for the nice work here!

    I’ve been messing the code you provided, and I’ve gotten it to the point where either: A.) the progress bar jumps straight to completed or B.) the progress bar bounces back and forth like many XP “progress bars”. “A” seems to happen when I have the PECL uploadprogress extension listed in my php.ini. “B” seems to happen when I comment those things out and just HTML_AJAX and friends do their thing.

    I’ve compiled the extension both by hand and from the “pecl install uploadprogress-beta” call, but both result in the same thing.

    I’m fairly new most of these web technologies, so I could be doing something wrong.

    My server is an Ubutu 7.10 box running the latest (for Ubuntu) PHP 5.2.1. Any thoughts?

    Thanks again!

  3. Joshua Eichorn Post author

    Ichibod:
    Sorry I didn’t reply earlier

    Without the progress extension you would expect the bouncing bar, there isn’t any data so the code doesn’t know when things are done.

    If things jump straight to complete then there is something broken. I’ll see about writing up a how to of install things on 5.2.x but i’m not sure when i’ll find the time.

  4. Daniel

    I’m wondering if all the patching done in PHP 5.2.1 broke the uploadprogress 0.3.0 extension. I was able to get it installed fine (PHP 5.2.5, centos) showing up in both the phpinfo() call and returning true from the extension_loaded(uploadprogress) call but when unloading a file I didn’t see either of the two files in /tmp (upt_%s.txt & upload_contents_%s by default). If you’ve been able to get this to work with PHP 5.2.5 Josh (or anyone else for that matter) I’d greatly appreciate your insight. In the meantime I’m back to my animated gif progress image ;)

  5. Dinesh

    I am running PHP 5.2 on Ubuntu server. I have installed php5-dev. I also ran the following command: perl install HTML_Ajax-0.5.2 and perl install uploadprogress-0.3.0. I have modified php.ini to set the temp folder. I have the move command and the file is uploading fine. I have restarted apache2 BUT the progress bar is going back and forth. Also I am not seeing the file byte progress or percentage. Am I missing something?

  6. Dinesh

    Correction: I meant to say “pecl install uploadprogress-0.3.0″. All commands ran successfully without any errors.

  7. Dinesh

    another correction: perl install HTML_Ajax-0.5.2 should be pear install HTML_Ajax-0.5.2

  8. Joshua Eichorn Post author

    Dinesh: make sure to get the latest progress bar code from svn and check your phpinfo screen tomake sure that the uploadprogress extension is getting loaded

  9. Dinesh

    Dear Joshua,

    Need your help desperately again. I am trying to echo an error to the brower if the tmp bytes_total is more than 300MB. Usually PHP only sends an error message after the file has been uploaded to the tmp folder and before it is moved. I want the client to get an error before the file is uploaded. I heard from some other blog that this is possible with the new uploadprogress extension and hooks. If yes, how would I modify your code to make this work?

    Thanks!

    Dinesh

  10. Joshua Eichorn Post author

    Dinesh: I haven’t heard of anything in the extension helping that, but there is a comment that the manual talks about that sets the max size limit in the browser where it should be enforced.

  11. Dinesh

    Joshua,

    Will that work for all the browsers? Also where can I find this manual? I have been looking for it.

    Thanks!

    Dinesh

  12. Joshua Eichorn Post author

    Dinesh: no clue about browser compat, but i think it does

    http://www.php.net/manual/en/features.file-upload.php

    from the manual -
    The MAX_FILE_SIZE hidden field (measured in bytes) must precede the file input field, and its value is the maximum filesize accepted by PHP. Fooling this setting on the browser side is quite easy, so never rely on files with a greater size being blocked by this feature. The PHP settings for maximum-size, however, cannot be fooled. This form element should always be used as it saves users the trouble of waiting for a big file being transferred only to find that it was too big and the transfer failed.

  13. Dinesh

    Joshua,

    Thank you! That seems to work but different browsers seems to come back with different error messages. For exampe: IE7 says Internet Explorer cannot display the web page. Firefox comes back with an error that reads The connection was reset. I guess I can use JavaScript to make the error more user friendly. The MAX_FILE_SIZE option i.e. maxFileSize in UploadProgressMeter.class.php is definitely a nice layer of security in addition to the maximum limits set in php.ini. Some JavaScript would be a nice finishing touch. I am working on that now. Thanks again!

    Dinesh

  14. Dinesh

    Joshua,

    Bummer! It looks like I need an ActiveX control or a perl script to grab the file size from the client side before the file is uploaded (on submit or on post). I read somewhere that PHP 6 might be able to get the file size prior to an upload. Any recommendations? Please help!

    Thanks

    Dinesh

  15. Dinesh

    Dear Joshua,

    I have made progress. The bar does not jump to 100 percent for smaller files but when I try a 695 Megabyte ISP file it jumps immediately to 100 percent. My post and max sizes in php.ini and in the script is set for 300 MB, so I can see what this may be happening, but is there a way to make it stick to 0 percent and give the client an error message saying that the file is too large to upload? I dont see why this cannot be done because as soon as the upload progress bar starts, the stats clearly show a total byte count i.e. the variable $tmp['bytes_total'] from the UploadProgressMeterStatus.class.php script. This means that it is getting the total byte count of the file before it is uploaded to the temporary folder. How do I echo an error to the browser. I have tried differnt approaches but it does not seem to work. Please help! Thanks.

  16. Joshua Eichorn Post author

    Dinesh You will need to update the javascript code to produce the error(have the PHP side add an error code to the returned information and check for it in UploadProgressMeter_Update)

    I’m starting to work on an updated version of the script that has better error handling but i’m not sure when it will be done. Feel free to email me (josh@bluga.net) if your interested in sponsoring some improvements to the code or if you have some work you’ve already done that you want to add back to the public version.

  17. Dinesh

    Joshua,

    I just increased the sleep time from 1 to 3 seconds and I changed if ($tmp['bytes_total'] < 1) to if ($tmp['bytes_total'] message within a div

  18. Dinesh

    The bar jumps to 100% for large files. I have tested it for a 104 MB file and it works fine, but when I tested it with a 499 MB file it immediately jumps to 100%. The problem seems to be the following section of the UploadProgressMeterStatus.class.php file: $tmp = uploadprogress_get_info($upId); What can I do to fix this behaviour?

  19. Joshua Eichorn Post author

    Dinesh:
    That methods comes from the upload progress extension, one option would be to upload the code to use the apc tracking methods and see if that fixes the problem.

    The other is too just detect the error and report an error message in that case.

    I’ve not seen any cases where the tracking info isn’t getting written out so i would guess comment:
    Some folks may be having issues if the upload_max_filesize ini variable
    is smaller than the file they are attempting to upload. I found that if
    the file I was attempting to upload was larger than the allowed size,
    uploadprogress_get_info() would always return null.

    is correct and fixes bigger then upload_max_filesize are causing problems.

  20. Dinesh

    Installing APC and enabling it to work along side the uploadprorgress extension seems to have broken the scripts even further. Now it jumps to 100% for every file, even small files. Futhermore, isin’t the overhead more when you use APC and uploadprogress at the same time? ….or are you recommending using one or the other….and not both at the same time? Error handling needs major improvement as the front end is designed for regular people and not programmers. Anyway, it’s back to the drawing board for me. Every little input and advice from you helps, so please keep it coming. Thanks again!

  21. Dinesh

    Joshua,

    Ok, thank you. I will try the APC approach also. For now, what I did was increase the post and max file size settings in php.ini to 1G. I am letting the UIploadProgressMeterStatus.class.php file to do the error checking based on a 512MB max file size limit and I am sending an error code to the UploadProgressMeter.js file for issuing an alert to the browser. That seems to be working fine for me. I just want to know how to stop the progress bar and remove any temporary files or cache when it finds an error. I would GREATLY appreciate a solution for this. This would solve most of my problems. Thanks!

  22. Dinesh

    Joshua,

    Basically along with my error checking I would like to halt or exit the script gracefully. Is that possible?

  23. Joshua Eichorn Post author

    Dinesh:
    I’m not sure what graceful options you have, the upload is happening in a different php process then the one checking status. So any upload cancellation is going to have to happen on the browser side, and I don’t know any way to cancel an in process form submission using javascript (which doesn’t mean its not possible, i’ve never really looked).

  24. Dinesh

    Joshua,

    What if you simply redirect the client or browser back to the home page or main form as part of the error checking. That would break the upload in progress correct? It seems to be working for me. I am also using the JavaScript stop function but it only works with Mozilla and not IE. I think I answered my own question. Thanks.

  25. Clint

    Hello,

    Where in all the code can I set the directory for which the uploaded file will go to? I can’t seem to find it anywhere, must be this head cold.

  26. Joshua Eichorn Post author

    Clint:
    You have to move the uploaded file at the end of the upload progress just like any other PHP upload script. There is a comment in the code showing you where to edit.

    See the php manual for instructions on how to handle the file upload, none of the upload process is changed

    Also you may want to check out the new version of this script, which has shows how to show the hidden iframe for debugging.

  27. Dinesh

    Joshua,

    Is there a way to show the name of the file that is being uploaded (including for multiple file uploads)? For example: bytes_total shows the total bytes, I was wondering if there is anything for the real file name and temporary file name? I am looking to put this in the UploadProgresssMeterStatus.class.php file.

  28. Dinesh

    Joshua,

    Ok, this is a 2 part question:
    ///////////////
    1) Going back to post number 222 on this blog, please note that in your own script file i.e. UploadProgressMeterStatus.class.php you are returning the message Upload Complete when the uploadprogress_get_info() is returning null so how would I detect the error and return an error message to the client in this case? Maybe I am missing something. Basically how would I differentiate between somebody attempting to upload a file larger than the upload_max_filesize in php ini versus a completely legitimate file that has completed it’s upload?
    //////////////
    2) It appears that the post_max_size limit is 1.99G in the php.ini file. I am unable to make it work with anything larger than that value. I get an error as follows: Apr 24 20:37:04 web1 apache2: PHP Warning: Unknown: POST Content-Length of 44 bytes exceeds the limit of -2147483648 bytes in Unknown on line 0
    //////////////
    Is there really a 1.99GB limit? Is there a way to upload files larger than 1.99GB?
    I’m sorry for asking tough questions and sorry if I am giving you a hard time, but I have searched for days looking for an answer and you are the only person that I can think of that may be able to answer these questions successfully.
    //////////////
    Thanks,
    Dinesh

  29. Joshua Eichorn Post author

    I’m not sure on #1, maybe a null after a good value is complete and a null before a good value is an error. You could track the status in the session.

    For #2 its doubtful that files over 2gb work (at least on 32bit oses) normally you have to do special stuff to go larger and i doubt thats been a priority. If you want the details you should ask on a general php list.

  30. Dinesh

    Joshua,
    ////////
    Re: #1: I am not sure I understand your response. I will need some more time to think about your response and research it. Re: #2: Would it be possible to bypass the 2 GB linux file limit for 32bit OS’es if I created a raw partition for the files?…or would the OS still impose that 2 GB limit on that partition? I know that this is off topic, but any help would be greatly appreciated. I read something about LFS support (large file system), but will that work for Apache/PHP also?
    ////////
    From what I read on the php list, you are right, it is not a priority, but I wish it was.
    ////////
    Thanks
    Dinesh

  31. Joshua Eichorn Post author

    Dinesh:
    I belive the problem is that you can get a null response for 2 reasons. The upload is complete or it failed. I was just suggesting storing that you’ve started a good upload in the session to differentiate.

    The 2gb limit shouldn’t be due to a OS limit on any modern system. But only apache 2.2 > support files over 2gb and I have no clue about the PHP upload processing code which is what really matter in this case.

  32. Dinesh

    Joshua,
    ///////////
    This may sound like a simple or even stupid question, but how would I store or capture that a good session has started? How would I do the error handling for a failed session? I will eventually figure it out as I usually do, but I hope I dont have to waste any more time with this. :-)
    ////////////
    By the way, I am running Apache 2.2, ext3 filesystem, php 5.2 and the latest ubuntu server with updates, but I am still unable to pass the 1.99GB barrier.
    ///////////
    apache2 -v says:
    Server version: Apache/2.2.4 (Ubuntu)
    ///////////
    php -v on command line says:
    PHP 5.2.3-1ubuntu6.3 (cli) (built: Jan 10 2008 09:38:37)
    ///////////
    PHP web environment says:
    PHP Version 5.2.3-1ubuntu6.3
    ///////////
    Will switching to a 64bit hardware and OS version be a guaranteed fix for this issue?
    ///////////
    Dinesh

  33. Joshua Eichorn Post author

    WXP:
    I’m not sure why you would want to make a sync request it locks the browser ui. But if you wanted to use the new version of this code and just update the call to use the sync version by removing the callback function.

  34. Kiqlo

    You can use swfupload to upload your files. If uses flash but the integration with JSP, PHP is effortlessly and immediate. I tried on the site Kiqlo and it works great.

  35. Albert

    SWFUpload is NOT reliable solution. I have been struggling with too many problems (random firewall, browser setting, …) can stop your upload at any time with infamous error number #2038. Besides Flash 10 breaks compatability!

  36. John

    I’ve adapted Rasmus Lerndorf’s php file upload progress meter. As long as you have php5 and APC, everything should be very self-explanatory.

    Click here to view the demo.

  37. Pingback: gaemon's me2DAY

  38. Elevatelocal

    Hello,
    Where in all the code can I set the directory for which the uploaded file will go to? I can’t seem to find it anywhere, must be this head cold.

  39. aneeq

    The code to upload file in PHP is very simple, but we need to understand the flow which is a below.

    1. Browse the file from a local system
    2. Upload to server
    3. Server keeps it on a temporary path
    5. Copy from temporary to permanent path

    Create a file upload form

    Filename:

    Note:

    * An enctype attribute of the tag has been specified.
    * This attribute specifies which content-type to use when submitting the form
    * We have used “multipart/form-data” to upload binary data, like the contents of a file, to be uploaded
    * If proper enctype is not provided, upload will not work.
    * File upload is a huge security risk so you must check what type of files are being uploade

    Create a file upload script (upload-file.php)

    This will upload the file to the specified path.

    Note:

    * The default file upload size using a browser is usually 2MB so files larger than this size may not upload. You will have to alter the file upload setting on the server.
    * You need to set write permission to the folder where file needs to be upload.
    * In our case, the “uploads” folder needs to have a 777 permission on a linux/unix server.

    Source:

    http://phphelp.co/2012/05/18/how-to-upload-a-file-in-php/

    OR

    http://addr.pk/a478

  40. Pingback: php – 파일 업로드 진행상태 : ProgressBar | 주저리주저리주저리..