Archive for the 'PHP' Category

Problems with GD: Solved

Tuesday, October 11th, 2005

In a recent project, I had the joy (make that pain) of using GD a lot. The project was an update to the a facial composite tool for Identikit. The new code isn’t live yet, its still in final testing, but its all done for me except for a bit of final bug fixing.

Anyway the app is written in Flash and integrates with PHP for things like saving files to the database or creating a jpeg for the user to save. So what I have is a bunch of transparent pngs that I composite together to make the final sketch. Just like you would do by hand if you had an old school version of this from little paper cutouts.

The problem I ran into with GD is it doesn’t repect an images transparency. So my basic test of this is, open an transparent Image and resize then output the image.

You take the starting Image (shown on and off a colored background so you can see the transparency)

Sample Image - Glasses with Transparent Lenses Sample Image - Glasses with Transparent Lenses

Next we resize it, some code for this is shown below:

<?php

// load in the source image
$im = imagecreatefrompng('sample1.png');
imagesavealpha($target,true);

// get its width and height
$imageWidth = imagesx($im);
$imageHeight = imagesy($im);

// were doubling in size
$targetWidth = $imageWidth*2;
$targetHeight = $imageHeight*2;

// create a target im
$target = imagecreatetruecolor($targetWidth,$targetHeight);
imagesavealpha($target,true);

// fill the image with transparency
$trans = imagecolorallocatealpha($target,255,255,255,127);
imagefill($target,0,0,$trans);

// copy with resize to the target
if (isset($_GET['resample'])) {
        imagecopyresampled($target,$im,0,0,0,0,$targetWidth,$targetHeight,$imageWidth,$imageHeight);
}
else {
        imagecopyresized($target,$im,0,0,0,0,$targetWidth,$targetHeight,$imageWidth,$imageHeight);
}

// output the image as a png
header('Content-Type: image/png');
imagepng($target);

?>

Output from this is shown below, first using resize then using resample

Resized version of sample image
Resampled version of sample image


As you can see the background is still alpha transparent but the lenses are now black. I hope im just missing some flag, but in that case someone should really add a makeBrokenGDWorkLikeIWouldExpect flag so I don’t have to find the right combination of them to make things work. After all if I shelled out to ImageMagick convert to do the resize it would just do the right thing.

The next step would be to do some compositing, but you wouldn’t see it anyway since the eyes go below the glasses and they are black.

Anyhow feedback on what im doing wrong here would be nice, or if this just doesn’t work what I need to make this a good bug report.

Any how this example doesn’t have the problems I had before, that just had a simple bug in it from me filling in the wrong order.

Anyhow I did have similar problems when working with the compositing code for this app, im guessing it was caused by a similar mistake, but i’d have to read through commit logs to be sure, and that doesn’t sound like much fun. Suffice it too say, the equivalent code using Imagemagick on the command line is a lot easier to understand though not much easier to debug.

Anyhow the second part item after resizing an image is adjusting its properties. Shading the image, playing with gamma works pretty well, flipping it if needed (this would be nice to have as part of the GD api but its quite easy to write, This manual comment has one), and then adjusting the transparency of the image (Imagecopymerge works well for that).

A sample output would be:
Eyes with Glasses

And for a quality comparison, the same thing done shelling out to composite
Eyes with Glasses

The main things I would take out of this is that the API for GD is way to low of a level for normal use, so you’ll want to look into a wrapper (there are some options in PEAR but none with a stable release). There are also quality issues with GD when making images larger, but I don’t want to see new features like cubic interpolation added until the api to use them is reasonable.

Code for GD Compositing

<?php

// create a target im
$target = imagecreatetruecolor(500,200);
imagesavealpha($target,true);
imagealphablending($target,true);

// fill the image with transparency
$trans = imagecolorallocatealpha($target,255,255,255,127);
imagefill($target,0,0,$trans);

// copy the eyes in place first
// load in the source image
$im = imagecreatefrompng('sample2.png');
imagesavealpha($im,true);
imagealphablending($im,true);

// get its width and height
$imageWidth = imagesx($im);
$imageHeight = imagesy($im);

// were doubling in size
$targetWidth = $imageWidth*2;
$targetHeight = $imageHeight*2;

imagecopyresampled($target,$im,2,26,0,0,$targetWidth,$targetHeight,$imageWidth,$imageHeight);
imagedestroy($im);

// copy the glasses on top of that
// load in the source image
$im = imagecreatefrompng('sample1.png');
imagesavealpha($im,true);
imagealphablending($im,true);

// get its width and height
$imageWidth = imagesx($im);
$imageHeight = imagesy($im);

// were doubling in size
$targetWidth = $imageWidth*2.2;
$targetHeight = $imageHeight*2.2;

imagecopyresampled($target,$im,0,0,0,0,$targetWidth,$targetHeight,$imageWidth,$imageHeight);
imagedestroy($im);

// output the image as a png
header('Content-Type: image/png');
imagepng($target);

?>

Code for ImageMagick compositing

<?php

header('Content-type: image/png');
passthru('composite -geometry +2+26 sample2.png -resize 200%  back.png - | composite sample1.png -resize 220% - - ');

?>

HTML_AJAX 0.2.1 Released

Friday, September 30th, 2005

I just released HTML_AJAX 0.2.1, if your running 0.2.0 you’ll want to upgrade immediatly since this fixes a bunch of bugs I added in that version.

The main changes are:
Bugs:
Remove debug message when throwing an exception
Fix problems with async calls
Fix broken content-type detection
Stop trying to run an init method when the init flag isn’t set
PHP required version moved to 4.1.0 which is what it should need

Features:
A basic debug class has been added, allowing you to write PHP errors to a file
HTML_AJAX.replace now works async

Also there is now a mailing list for HTML_AJAX.

HTML_AJAX 0.2.0 Release

Tuesday, September 27th, 2005

HTML_AJAX 0.2.0 is out. There were a number of changes most of them on the JavaScript side.

The code is now released on the lgpl so thoses of you with GPL apps should feel comfortable using it now.

I don’t have time to write another tutorial showing off the new changes but if your looking a good place to start is with the new HTML_AJAX_Request class in JavaScript. All AJAX requests happen by creating an instance of this class and then running it through HTML_AJAX.makeRequest. If your looking to integrate your own complex widget this is the place to start.

The changelog has some more information.

Also im hoping to get the PHP side fully stabilized soon so we can move to a beta. If anyone has time I would appreciate an API review, mainly on HTML_AJAX and HTML_AJAX_Server.

Finally i’d like to drop the PHP dependency to something lower then 4.3.0, if you’ve used HTML_AJAX on 4.2.x or something else let me know. Once i’ve had confirmation it works i’ll lower it.

The roadmap has also been updated

azPHP September Meeting

Monday, September 26th, 2005

The meeting is tomorrow 7:00pm at Walt’s TV in Tempe, be sure to invite anyone you know that might be interested.

Were having two new things at this months Meetings. Eric Thelin will start the meeting out giving us an overview of his what he found at the recent Microsoft PDC conferences. The other new item item is our first Roundtable.

Also we have a great presentation about Database abstraction by Jon Roig.

Meeting Timeline –
Tuesday September 27
7:00 - Microsoft Update by Eric
7:15 - DB Abstraction with PEAR::DB Jon Roig
8:00 - Roundtable: PHP Architecture Members: Josh, Eric, Brock, Travis
8:45 - QA Session
9:30ish - Social Hour(s) at Old Chicago

Round Table Questions:
What are your thoughts on class/function namespacing ie (PEAR style HTML_AJAX)
Do you build frameworks or libraries
How do you manage external code (PEAR stuff, adodb, smarty etc)
How do you manage config information

I hope to see lots of new faces

azPHP August meeting wrap up: Beginners Guide to PHP + a Short Introduction to PEAR

Thursday, August 25th, 2005

azPHP had its August monthly meeting on tuesday. We were trying a new approach and a beginner and a more advanced presentation in the same night. We had a lot of new people there so I guess it worked.

a Beginners guide to PHP

Alex Dean gave a Beginners guide to PHP presentation covering interaction with data from the browser. He covered the basic super globals $_GET, $_POST, $_SERVER, and $_COOKIE. He also made a sample form and showed how to use it. The plan was also to cover using mysql but sadly we ran out of time. I think for future introductory presentations will use handouts as well as slides/example code that way people won’t have to spend so much time writing notes.

Alex did the presentation in a interactive manner writing code as he explained the topics so there is no slides but you can view the sample code. If your looking for other beginner resources the tutorial in the PHP manual is always a good place to start.

a Short Introduction to PEAR

I the presentation about PEAR, its was a shorter presentation since the first one went a little over its planned time. I tried to focus mainly on what PEAR is and why people should use it over something like hotscripts.

The slides from that presentation are available in my archive.

Towards the end of the presentation I mention channels and phar two new PEAR 1.4.0 features, but the slides don’t give any details. Greg Beavers blog is a good source of information about both topics.

Greg’s introductory post to using a phar for the pear installer.

PEAR manual information about channels.

I also found someone to take over setting up the new azPHP website, so hopefully will look a lot less dead a week from now.

AJAX - PHP presenter in Boston

Thursday, August 25th, 2005

I was contacted by the Mark Withingon, founder of the Boston PHP Users group today, there looking for someone who could do a presentation about HTML_AJAX for them in November. Now I would love to fly there and help them out, but its not going to happen, so the best I can do try to find someone else.

If your interested let me know and I’ll and get your in touch with Mark. I’ll be doing a similar presentation for my local PHP users group that same month so we should be able to share slides and preparation effort.

AJAX Article in php|arch

Wednesday, August 24th, 2005

A new article in php|architect is out, and an article I wrote about AJAX is in it.

The article gives a good introduction to AJAX as a concept and then moves into practical use.

The issue looks like a good with an article about PEAR 1.4 channels as well.

The PHP License

Tuesday, August 23rd, 2005

As you might have noticed HTML_AJAX is licensed under the PHP license. It is mainly because it borrows some JavaScript for JPSpan that was released under this license. It has been brought to my attention that there are a number of conditions in the license make its distribution problematic.

The problematic items (at least from a Debian packaging prospective are)

3. The name “PHP” must not be used to endorse or promote products
derived from this software without prior written permission. For
written permission, please contact group@php.net.”

4. Products derived from this software may not be called “PHP”, nor
may “PHP” appear in their name, without prior written permission from
group@php.net. You may indicate that your software works in
conjunction with PHP by saying “Foo for PHP” instead of calling it
“PHP Foo” or “phpfoo”

6. Redistributions of any form whatsoever must retain the following
acknowledgment: “This product includes PHP, freely available from
http ://www.php.net/”.

Generally these terms don’t apply well to stuff that isn’t PHP.

Anyhow how have other people approached this problem. For phpDocumentor im looking towards just doing a dual license with PHP and BSD, just because its been PHP licensed for so long. If I can get Harry to relicense the code I used I might just release HTML_AJAX on the BSD license for the next release, otherwise I guess im stuck unless I finish up all the JS refactoring I have planned.

Update:
Charles Fry wrote up Debian’s concerns and sent them to pear-dev if you want more details on why the license is a problem.

AZPhp Tomorrow

Monday, August 22nd, 2005

Tomorrow is the 4th tuesday so it will be soon time for an evening of PHP fun.

The main presentation will be a “Beginners Guide to PHP” by Alex Dean so make sure to invite anyone you know who is interested in getting started with PHP.
Along those same lines I’ll be giving a short Introduction to PEAR, and if there is some extra time going over the plans for the new website.

You can get the location from the groups website (yes were working on one that doesn’t suck), and as always if you have any questions, the mailing list is a great place to start.

The Schedule:
Tuesday August 23
7:00 - PHP news for 15 minutes
7:15 - Beginners Guide to PHP by Alex Dean
8:00 - Introduction to PEAR and the Group Website by Joshua Eichorn
8:45 - QA Session
9:30ish - Social Hour(s) at Old Chicago

Hope to see you all there
-josh

Website Usage Visuationzation and AJAX Tricks

Thursday, August 18th, 2005

I’ve been working on a small website stats project, mainly because webalizer et al don’t meet my needs. The actual implementation is mod_log_sql + a bunch of php/sql script to transform that to a star schema and then some php scripts for visualization.

Things are finally getting to the point of being useful, I have enough of the dw built out that I can answer basic questions about my site with a little sql fun, plus I have one good report. Now this report is where the AJAX tricks come into play, its a trick because were loading in data asyncronously, but its a trick because its done by changing the src of imges instead of using XMLHttpRequest. There is also a lot of neat JavaScript fun happen which I use to put up a loading screen while the images are being generated.

A look at the Report

Anyhow lets get a look actual report so you can see whats going on (normally I would post a link but 500 hits on this report would kill my server the main query takes 5-10 second, which can be optomized but i haven’t bothered). The image below is the starting page of the report, it has a list of my top blog articles this month with the date they were actually posted (well showed up in my logs) and there total hits. There are also 4 blank graphs with a default placeholder graphic.
Report Screenshot at start

Now if you click on one of the links in the table the 4 graphs will be updated with drilldown information about that page. The Images are generated with a query and then some processing with Image_Graph so they take anywhere from half a second to 2 to update, they also update at different times. To make things more user friendly I added a loading div that covers each image while its loading. This is shown in the screenshot below.
Report Screenshot during Load

After a second or two, the graphs finish loading and the loading screens disapear. This leaves you with a nicely updated report. Another nice effect (at least in firefox) is that the graphs stay cached as you click around between the different drilldown screens, but will update after a reload of the page. This makes for a snappy experience if you want to go back to something you’ve already seen.
Report Screenshot load finished

Making it Work

I’m not going to go into the details of the actual sql and Image_Graph code (contact me if you want to talk about it) but I will cover the JavaScript code that does the loading.

The way it works is:
You have a hidden javascript div that is a template for the 4 loading divs, it can contain anything you want (I’m lazy so mine just says loading).

On page load a setup function finds all the graph images (there are all in a div with the id of graphHolder) loops over the images and creates a loading div by cloneing the template. The new clones are then added to the same container as the images and absolutely positioned so they cover them. Since the template has display:none you don’t see the loading divs yet.

When we click on a link where actually using a javascript: url to call the updateGraph function, this function sets all the loading divs to a display of block so they cover the images and then it sets there new src.

And finally the Images have onload handlers hard coded on them that call loadingDone passing in this. Loading done then sets display back to none.

Something to play around with

So below is a real quick example so you can see if for yourself.

Code of Example:

Code of php script for making the color blocks with a nice delay.

<?php

require_once 'Image/Color.php';

sleep(2);

$im = imagecreatetruecolor(300,300);
$color = Image_Color::namedColor2RGB($_GET['color']);

$bg = imagecolorallocate($im,$color[0],$color[1],$color[2]);
imagefill($im,0,0,$bg);

header('Content-type: image/png');
imagepng($im);

?>

Wrapping it up

The great thing about AJAX is its more AJ then anything. If you have JavaScript there is no shortage of ways to load new data. Oh and on an unrelated note I’m writing an AJAX book, it should be some time in Jan, and I should have been writing a chapter of that instead of messing around with this :-). I’ll blog with some more details about that some time this weekend.

AJAX Hello World with HTML_AJAX

Wednesday, August 17th, 2005

This is a continuation of my AJAX Hello World series, in my earlier posts I covered sajax and JPSpan. In this article i’ll cover how to get a basic AJAX appliction up and running with HTML_AJAX. If you haven’t figured it out yet im one of the author’s of HTML_AJAX. The application in question is the same simple app as the other examples, it has an input box for adding random strings and an button to add a random one to a div.

When your using HTML_AJAX you have more choices then with either JPSpan or sajax. But to keeps things simple were going to use a setup similar to how we used JPSpan, an external server page that generates a JavaScript proxy which is included by our html page that does the actual action. HTML_AJAX could also be used sajax style with the proxy and server code generated inline, but I don’t recommend this usage since it remove your ability to cache the generated JavaScript.

Installing HTML_AJAX

This is nice and simple its simply:
pear install HTML_AJAX-alpha
If you don’t have pear setup on your server just follow the installation instructions in the PEAR Manual.

Setting up the Server Component

An HTML_AJAX server page is generally very simple, it creates an HTML_AJAX_Server instance, registers all the classes you want exported and handles incoming requests. The incoming requests can be three different types:

  • Client library request: query string includes ?client=all (can also be each of the seperate component parts)
  • Generated stub requests: query strings includes ?stub=classname (all keyword can also be used)
  • AJAX Requests: query string includes ?c=classname&m=methodname

Client and Stub requests can be combined into the same one, but remember there is a tradeoff when doing that. You’ll have less roundtrips to the server but generated stubs are more likely to change so they can hurt your cacheability. You also want to be careful about using stub=all since the stub for 10 different classes can be quite large. The next release will also allow for combining multiple classes in a stub request by seperating them with a comma like so: stub=test,test2. Finally a note on class names, in php4 they will be lower case since thats what PHP returns, if you would like to have case or need to guarantee php4/5 compatability you’ll want to use some optional paramaters to registerClass, the first is the name to register the class in JavaScript as, the second is an array of methods to export.
A basic server is shown below.

<?php

session_start();

require_once 'HTML/AJAX/Server.php';
require_once 'HelloWorld.class.php';

$server = new HTML_AJAX_Server();

$hello = new HelloWorld();
$server->registerClass($hello);

$server->handleRequest();

?>

A server like this works fine for small scale projects, but it might not be a good choice for something larger since you could be creating 20 class instances just to serve a request on one of them. Any custom setup also has to be done for each class. HTML_AJAX provides a way to manage this by extending the server class and adding an init method for each class. A server that does this for the same HelloWorld class is shown below.

<?php

session_start();

require_once 'HTML/AJAX/Server.php';

class AutoServer extends HTML_AJAX_Server {
        // this flag must be set for your init methods to be used
        var $initMethods = true;

        // init method for my hello world class
        function initHelloWorld() {
                require_once 'HelloWorld.class.php';
                $hello = new HelloWorld();
                $this->registerClass($hello);
        }

}

$server = new AutoServer();
$server->handleRequest();

?>

PHP Application Code

In the server examples above you’ll notice were including a Hello World class. This hasn’t changed from the JPSpan example. The only thing of note is that makes heavy use of session, many AJAX apps will share this same heavy use, so make sure your server setups can handle that.

<?php

// our hello world class
class HelloWorld {

        function HelloWorld() {
                if (!isset($_SESSION['strings'])) {
                        $_SESSION['strings'] = array('Hello','World','Hello World');
                }
        }

        function addString($string) {
                $_SESSION['strings'][] = $string;
                return $this->stringCount();
        }

        function randomString() {
                $index = rand(0,count($_SESSION['strings'])-1);
                return $_SESSION['strings'][$index];
        }

        function stringCount() {
                return count($_SESSION['strings']);
        }
}

?>

The HTML that ties it all together

To finish up our app we need to create the HTML that ties it all together. This is going to have two main parts, the main JavaScript logic, callback function for our async requests and a fake form. The call the form fake, because there is no form tags, were just powering everything off onclick event handlers on buttons. At some point in the future HTML_AJAX will support taking a normal form and magically turning it into an AJAX one but for now that thought is just in my head.

The page starts with normal HTML boiler plate and an include of our client library and the stub hello world class. They are done all in a single class since that makes sense for this example, if I had 10 classes it wouldn’t.

Next we build the callback hash, it has a method for each method in PHP, they will be called from the results from an async AJAX call. In this example each call back is really simple it either updates an element using innerHTML or appends to one. You could also use a normal JavaScript class instead of a hash, but thats only useful if you need multiple instances.

The next step is to create an instance of our helloworld class and create a helper function. When we create an instance of our proxy class we pass in our callback, if we didn’t the instance would work in a sync manner, which can be useful, but in this case would make for a choppy ui. The helper function do_addString isn’t anything exciting, it just clears out the textbox after we’ve submited its current value.

Finally we have the html body of the page, it contains our buttons with onclick event handlers that call remote functions. It also contains our output divs.

Trying out the sample

If you got this far you’ll want to try out the Hello World app were building. If you compare it against the JPSpan or sajax version you won’t notice many differences. The only visual one that should show up is loading notice in the top right corner. Since providing user feedback is such an important part of making a usable AJAX application HTML_AJAX includes this basic loading symbol out of the box. If you just want to customize its looks you just need to create an element with the id of HTML_AJAX_LOADING. Its shown by setting its display style to block, and hidden by setting display to none everything else is left up to you so make sure to take care of any positioning and setting its display to none so its hidden by default. If you want to do somthing more complex you can override HTML_AJAX.onOpen and HTML_AJAX.onLoad check out the current methods to see hows its done.

Finally you might notice some JavaScript errors if you try to make a lot of quick requests, like JPSpan HTML_AJAX only provides for one concurrent request at a time. This will change in the future giving you a number of different options but for now you can either keep those from happening at the application level, or just add a error handling function to HTML_AJAX.onError and ignore the call in progress ones.

You can also download the project and try it out on your own server, its available as a tar.gz or a zip.

Where to go from here

Now that you’ve had a look at HTML_AJAX you might be wondering how to learn more about it. For the moment your options are pretty limited. You can check out the examples dir (in the docs dir in pear or online) or you can ask questions on this post. You might also find the source to be useful, for now the newest version lives on my svn server.

Feed back is appreciated as I’ll use that to decide what features to add to HTML_AJAX and what to fully document next, feel free to post comments or trackbacks even if you have a lot to say.

Update:
This example won’t work correctly in PHP5 since the methods will be exported with CASE, take a look at the generated JavaScript if you don’t understand what i mean (auto_server.php?clalss=helloworld). See Comment #40 for your options to fix things.

Say Hello to HTML_AJAX

Thursday, August 11th, 2005

HTML_AJAX had its first PEAR release today. Its the same version as was voted on so there is no new code but you can use the PEAR installer to get it.

Once you have it installed you’ll find the examples in the docs dir to be quite helpful, on my server thats /usr/share/pear/docs/HTML_AJAX/examples. These examples are also available online: run, view code.

I’ve started a basic website for the project, it includes a development roadmap, so if you want to help out, or just want to know whats happening next thats a great place to start.

HTML_AJAX has two different main use cases, standalone and proxy.

Now in proxyless/standalone mode you can use just the JavaScript library making requests to pages and working with its results however you want.

To accomplish this basic usage you either need to create a instance of a server class that will server the JavaScript libraries, or copy them into your web root. These js files are located in your pear data directory, on my server thats: /usr/share/pear/data/HTML_AJAX/js.

Setting up a server that will handle client requests is easy, a basic one is shown below:

<?php

include 'HTML/AJAX/Server.php';

$server = new HTML_AJAX_Server();
$server->handleRequest();

?>

If you save that as server.php you can include the needed JavaScript libraries like below:

Once you have the libaries pulled in you easily make various AJAX calls, the main usage for proxyless operation is to pull down some new content and put it into your page using innerHTML. Any asyncronous example of this is shown below:

A sync version of grab is also available, just don’t pass it a callback function. This is even easier to use but has the disadvantage of blocking while the download happens. In some circumstances this can useful since your already in a callback, but you have to be careful when using it. Besides the grab method there is also a replace method it takes an id and replaces its innerHTML with the results. Finishing up the standalone api there is a call method which allows you to call methods on classes registered with the server. Registering classes is shown in the proxy section. A sample async call is shown below:

Call can also be used to do sync calls just pass false in instead of a callback function.

Moving on from the basic standalone api HTML_AJAX offers generated proxy classes. You start this usage by registering a class with the server. The class will be introspected and public methods will be exported (methods starting with _ are considered private).

<?php

include 'HTML/AJAX/Server.php';

class test {
  function rot13($string) {
    return str_rot13($string);
  }
}
$test = new test();

$server = new HTML_AJAX_Server();
$server->registerClass($test);
$server->handleRequest();

?>

After that you need to include the generated proxy class. After that you can create an instance of that, when you do it you pass in a callback class. An example of this calling the rot13 method is shown below.

There is some other optional functionality around as well, the examples show documentation of pretty much everything. I’ll write more formal documentation as I have time. If you have any questions feel free to ask them on this post, and i’ll either reply in the comments or in a new post if there is enough of them.

Frankfurt PHP Users Group AJAX Presentation Wrapup

Thursday, August 11th, 2005

I just finished a short presentation for the the Frankfurt PHP users group . I’m not sure thats its the best presentation i’ve ever given since I tried to do mainly a overview of the tech as well as a bit of talk on how it relates to flash.

Might have been easier just to go the normal route and show a bunch of code :-). Anyhow the technology worked pretty well for the slides but showing web pages didn’t work right at all.

I have the slides online, if your more interested in getting started playing around with AJAX one of these AJAX tutorials might be a better fit.

Also HTML_AJAX is approved and the first release has been made, i’ll be getting more docs up shortly.

Cake, mp3act and lots of other AJAX Updates

Tuesday, August 9th, 2005

I’ve been slowly updating my AJAX resource list, I’m down to about 20 items on my incoming list now to review. The biggest things i’ve found recently are the Cake PHP framework, and mp3act.

Cake is a PHP framework based on Rails, I like frameworks of that style because they don’t need any configuration files, its 100% code driven. Cake seems to have a focus on making things easy for the User, and its AJAX support is no exception, the helpers class looks like it really cuts down on the amount of code needed to get things done. There is a neat example page that was created to shown off all the AJAX stuff that cake can do. If you goto the website an notice its quite parse, take a look at the trac site, it has some pretty good docs linked off the menu bar on the right side, the site was moved recently and it seems that a website redesign is lagging behind.

mp3act is a one of those streaming music servers, its based on PHP and sajax (which wouldn’t be my choice but I guess its been around for awhile). It really shows what you can accomplish if you don’t worry about browser compatability and just code for firefox. That being said, I don’t see anything there doing that couldn’t be done on IE is well. Which brings me to an interesting point that I read in the Alex’s OSCON slides, you don’t want to be writing JavaScript yourself. I mean feature-wise JavaScript isn’t a bad language, it has prototype based OO and dynamic closures, and a decent programming environment in Firefox, but there is still IE to worry about (and Opera, and Safari). Each browser has a bug here, or something not completely implemented. Working around these bugs isn’t something you want to do on your own, by using Open Source libraries we can share in the work and the fixes. With a little bit of luck will all be able to create powerful AJAX apps without having to go the mp3act route.

I’m going to keep this pattern up of only posting update overviews on a weekly or bi-weekly basic since I don’t like clogging up my blog with tons of little update posts. If you want to see whats happening on a daily basic I’ve added a recent updates page. I’d like to add an rss or atom feed for that as well as the rest of the resource pages, but I don’t want to bother with learning any of the standards. If anyone knows of a feed writing class that works in PHP4 let me know.

Also I’ve thought about injecting the updates into my regular feed (not the one that goes to planet-php), but im not sure how reasonable that is to do in Wordpress or how kosher it is to inject almost blog content into my blog feed. Any thoughts on this would especially be appreciated.

Libraries

Behaviour : Using CSS selectors to apply Javascript behaviours
Small JavaScript class that can be used to apply behaviours to documents in a non intrusive way
CakePHP
Rails port to PHP, provides the same type of AJAX tie-ins

Articles

Will AJAX help Google clean up? | CNET News.com
General AJAX coverage focusing on Big players like Google, MS, Macromedia/Adobe and how AJAX will affect the market for things like Flash and XAML
Christian Cantrell: Deep Linking in Flash and AJAX Applications
Article which discusses problems with linking to a page created using AJAX, different techniques and there problems are covered
PXL8 - IFrame Remote Scripting
Article covering howto load data from a remote server using IFrames
Using the XML HTTP Request object
Article covering the ins and outs of using XMLHttpRequest on various browsers, various requests such as GET POST HEAD are covered

OSS Applications

mp3act - Streaming MP3 Jukebox AJAX Web Application
A PHP GPL application that provides a Streaming MP3 Jukebox, only Firefox and Safari are support.
Broken Notebook AJAX spell checker
A php project that adds spell checking to a textarea using the pspel/aspell extension and cpaint

Blogs

AjaxDeveloper.org
AJAX blog aggregator, each posting contains a short quote and some commentary

Examples

Cake Framework AJAX Demo
Demo of all AJAX Helpers in Cake PHP Framework

Presentations

Building Dynamic Web Applications with AJAX
Slides to presentation given at the AzPHP Users group in April ‘05, gives an AJAX intro and shows a demo app built with JPSpan
Learning AJAX OSCON ‘05 Tutorial Session
Good overview of AJAX and then building an Auto-save text area with Dojo and Prototype, sample code linked from slides

Blog Hacking

Monday, August 8th, 2005

I use WordPress for my blogging needs, and though I think it has some problems mainly poorly written database queries and lack of abstraction of core elements, it is quite powerful.

One feature I use a lot are Pages, these are really just a basic CMS built into WordPress. Pages are a great integration point for custom code like my AJAX resource list, since they can have a custom template type. If you already have your own custom theme setup creating new Page templates is easy. Besides including the external code that renders out the resources I also added comments to those pages, I didn’t see this well documented anywhere but it turned out to be easy. You just add the following snippet below to the template, just make sure to put it inside the content div if your still following the normal template layout.

<?php

< ?php comments_template(); ?>

?>

If your hacking on WordPress the place to start is at its wiki, Codex. The wiki doesn’t have everything document but it has the basics. If your looking for plugins there is a nice interface to download them directly from svn, it even picks up tags so you can see all the releases.

The plugins that im using are:

  1. Syntax Highlighter Enscript - Highlights any type of code that enscript knows about
  2. Most Commented
    - Shows a list of posts with the most comments on my sidebar

I’ve tried some other plugins over time, but I tend to to use as few as possible since they make the upgrade process more painful since most a quick hacks and aren’t updated on a regular basic. The biggest piece thats missing for me is plugin for manging photos, I don’t want anything fancy just something that works realibily lets me browse a directory upload, and make thumbsnails. Oh and it has to work on wordpress 1.5, I used ImageManager before the last upgrade, but its integration is kinda hackish, plus its overkill for my needs.

HTML_AJAX Call for votes

Thursday, August 4th, 2005

I called for votes on the HTML_AJAX proposal on tuesday, it has 13 +1’s already so it looks like it will pass.

The voting period ends on the 10th so there is still plenty of time for you to review the code and vote on it if your a PEAR developer.

Moving forward on development im looking for feedback (prototype implementation would be great too :-)) on the big features im planing to add.
They are:
Call Queuing and ordering of Async calls
Call combining (if you have 10 items in your queue send them all at once) for both sync and async calls
Some sort of widget api.

I’m really not really sure what the widget api will look like, one though is to define a set of events and implement PHP and JavaScript base classes for each of them. I guess both sides will also need an api that allows you to dispatch those events as needed. Anyhow if anyhow ideas on how this should work let me know.

There is also some boring work to do:
The request JavaScript class seems a bit pointless so it would be good to slim it down even more or hopefully just replace it
Move the HTTPClient init out of each dispatcher class on onto the HTML_AJAX static class, that way it will be easier to add an object pool and put other rules about how many clients we have sitting around.

Javascript Goodness

Tuesday, August 2nd, 2005

Clay over at Pearified has been packaging popular JavaScript libraries an making them available over a pear 1.4 channel.

To install them you need to run:

pear channel-discover pearified.com

Then install the packages like

pear install pearified/JavaScript_Behaviour

Unless you have your data_dir set to something in your webroot you still need to expose the javascript file somehow. Your options are, writing a PHP script that will readfile the files (useful if you want extra control over caching, want to combine multiple libs together in an app specific way, or want to add gzip compression). If your on a Unix system you can symlink the files to your webroot, or you can use an Apache alias (or your webservers equivalent).

Using Eval in PHP

Monday, August 1st, 2005

PHP contains an eval function, and since it lets build PHP code at runtime it allows for some very neat tricks, such as creating mock objects or soap proxy classes at runtime. Though eval can be useful that doesn’t mean its not exteremly dangerous, in this post im going to talk about when I think eval should be used, and some of its security concerns.

Eval’s Inherent Security Risk

Eval by its nature is always going to be a security concern. You taking a string from an external source and bringing it into your PHP script, you can think of attacks of this nature to being equivalent to SQL injection though they can generally cause a lot more damage ($GLOBALS generally contains your DB password and PHP has lots of filesystem functions). Now proper escaping and data cleaning should mitigate these risks but its easier to just avoid them whenever possible.

Uses of Eval

Eval has 3 primary uses, providing compatability between php versions, generating class definitions and logic at runtime, and call functions with variable names. Now when we look at these 3 use cases the actual when to use eval choice breaks down like this: using eval for compat is great, for logic is good if there is no other way, and NEVER to call functions.

Using Eval to provide Compat

This is a pretty straight forward use case, based on phpversion check you either use eval to create a function or execute some code at runtime. Most compat can be provided but without eval since a function definition inside an if statement is conditionally declared, but in cases where a new PHP5 keyword is used eval must be used to prevent PHP4 compile errors.

PHP_Compat in PEAR provides a number of compat functions, its clone method shows using eval to prevent compile errors, whiles its call_user_func_array method shows using eval to solve a problem when there is no other solution.

Using Eval to generate class/function definitions at runtime

This is an area where you end up using Eval because there is no other solution, but before you make the choice you might want to look at your design. For example you might think that using eval to run say store validation rules in a database and combine them automatically is a good idea, but this same solution could easily be done though a function naming scheme and just storing the name of the rule in the db. There are lots of cases like this, I tend to make the decision like this, if the solution can be done any other way try that before looking to Eval.

A good example of when eval is required is generating a proxy object from WSDL data (you could actually write the data out and include it as well but that only makes sense if there is an integrated cache, and has the same security problems). Without eval there is no way to create a class definition at runtime, so what you do is pulldown the xml definition of the class, generate a PHP class from that and then include it.

This brings up lots of security concerns, the biggest being that your taking data from a non-trusted source and generating PHP from it. In this case its alleviated by the fact that the actual body of the functions is just a stub that calls another worker method, in fact in PHP5 you could use __call instead of this approach (though im guessing generating the proxy is faster). So this means you can do very strict validation of the function, an example function is show below (note this isn’t the real from WSDL.php its just example code).

<?php

$function = "
// generated stub for the $funcName method
function $funcName() {
    soapCall('$funcName',func_get_args());
}
eval($function);

?>

As you can see we have just one variable to worry about cleaning, its a string and it can only be the characters allowed in a function name. Thats an important way of looking at cleaning data your going to eval, you could say it can’t contain a ; or a -, but you might miss something important, its always easier to make a whitelist rather then a black list. An example of cleaning the input is show below using preg_replace, I like to use preg_replace for this sort of thing, since regexs allow for an easy whitelist and you can decide to either replace offending characters with a default placeholder like an _ just remove them, or keep the same basic syntax, switch to preg_match and throw an error when an offending character is some instead of just cleaning it out.

<?php

$funcName = preg_replace("/[^A-Za-z0-9_]/","",$funcName)]

?>

Note: I couldn’t find a list of allowed characters in the PHP manual if anyone knows where its located please let me know
Note: While this escaping prevents security problems, it doesn’t prevent all errors since it will let a function name starting with a number though and thats not allowed.

Now if you were using user content to generate more then just function stubs you’d have to be more careful about cleaning and escaping your input, any strings should always be escaped with addslashes and any variable names with similar escaping to the code above.

Using eval to call variable functions

This is the simple case just never do it, PHP offers a couple of different ways to call functions with the name of a variable so unless you stuck on PHP < 4.0.4 eval shouldn't be needed.
For pretty much any use you can just use call_user_func_array, but you might also find just using variable functions is easier in some cases, the same syntax can also be used for objects.

The speed of eval

Besides security concerns eval also has the problem of being incredibly slow. In my testing on PHP 4.3.10 its 10 times slower then normal code and 28 times slower on PHP 5.1 beta1. This means if you have to use eval, you should avoid using it inline in any performance sensitive code. Any easy way to cancel the performance penality is to create a function in eval and just call that, now an extra function call does have some performance overhead but its pretty small and depending on the design can be non-existant since you would be calling some function anyway.

Raw Speed Data:

[jeichorn@bluga ~]$ php -v
PHP 4.3.10 (cli) (built: Jun 13 2005 21:25:09)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies

[jeichorn@bluga ~]$ php evalspeed.php
Eval: 1000000 times took 8.7586531639099
Same code not eval:  1000000 times took 0.99519085884094
Eval: 1000000 times took 2.1635251045227

[jeichorn@bluga ~]$ php5 -v
PHP 5.1.0b1 (cli) (built: Jun 13 2005 12:41:46)
Copyright (c) 1997-2005 The PHP Group
Zend Engine v2.1.0b1, Copyright (c) 1998-2004 Zend Technologies

[jeichorn@bluga ~]$ php5 evalspeed.php
Eval: 1000000 times took 8.6683559417725
Same code not eval:  1000000 times took 0.32248520851135
Eval: 1000000 times took 1.1273620128632

Simple Speedtest script:

<?php

/**
 * Simple function to replicate PHP 5 behaviour
 */
function microtime_float()
{
   list($usec, $sec) = explode(" ", microtime());
   return ((float)$usec + (float)$sec);
}

$rounds = 1000000;

$start = microtime_float();
for($i = 0; $i < $rounds; $i++) {
        eval('$b = $i;');
}
$end = microtime_float();
echo "Eval: $rounds times took ".($end-$start)."n";

$start = microtime_float();
for($i = 0; $i < $rounds; $i++) {
        $b = $i;
}
$end = microtime_float();
echo "Same code not eval:  $rounds times took ".($end-$start)."n";

$start = microtime_float();
eval('function test($i) { $b = $i; }');
for($i = 0; $i < $rounds; $i++) {
        test($i);
}
$end = microtime_float();
echo "Eval in function: $rounds times took ".($end-$start)."n";

?>

Security Reference

I’m currently looking for articles and reference about using eval securely, please send me links if you know of anything.

Tag Filters (Update on Query Problems)

Tuesday, July 26th, 2005

From the couple responses I got to my posting about Query Problems it looks like there is no great solution to tag filtering. It does look like using a subquery for each tag you want to filter by works reasonable well up to at least hundreds of thousands of tags, so that approach should work just fine for me.

The actual filtering is in my AJAX resources, and is currently just on there attributes (language they are written in, etc) not resources other tags. I think to really make the filtering work nice i’ll use some sort of multi link approach were you can add the filter as an or link, as an and, or view view that tag. I’ve also been thinking about adding some AJAX to the page just to show it off, but I’m not sure its really helpful to usability besides possible faster loads times, I’ll have to think about it a little more.

An example of why the filtering was added is now you can see all the PHP AJAX libraries, where before all the Ruby or Perl ones would be mixed in.

Query Problems

Monday, July 25th, 2005

I’ve been attempting to add some filters to my AJAX resource list. I’ve made an attempt an added them, but they don’t work the way I would like. So lets say your on the library list, a filter that would make a lot of sense, would be to say show me every library thats written in PHP and Provides AJAX. Now if you goto that link you’ll get every library thats written in PHP or Provides AJAX. Now I think the UI worked out pretty well but the or thing stinks, and that all comes down to database queries.

So the database is 3 tables:

Bookmarks
  bookmark_id
  user_id
  url
  ...

bookmark_tag
  bookmark_id
  tag_id

tag
  tag_id
  user_id
  tag

So I end up with a that is basically:

select b.*
  from
bookmark b
  inner join bookmark_tag using(bookmark_id)
  inner join tag using(tag_id)
where
  tag.tag = 'library' and
  b.bookmark_id in(
    select bookmark_id from bookmark_tag inner join tag using(tag_id) where tag.tag in ('language:php','provides:ajax')
  )

This works great but I export for multiple filter tags means or, since they are in an in list, or subquery for filter which seems like it would be horrible for performance.

Am I just missing a simple solution to the problem or is a subquery for each filter the only option?

Btw: I’m using mysql 4.1.something

This circle expands additional navigation
You should really buy my AJAX book.
You'll learn not only the technology you need for implementation but also an understanding that will help you get the most from AJAX.
You'll also have my eternal thanks :-)