There and Back Again

Over the last couple months I had a chance to help create a great new product for Stumbleupon. Su.pr lets create short urls, publish directly to Facebook and Twitter and gives you great tracking and statistics around them. Su.pr also helps you get more traffic by making it easy to get your content in front of the Stumbleupon community.

Last week we launched our API which lets you integrate shorten and post functionality into any site or application. Today we are adding the ability to use your own domain for shortening. This lets you have urls like http://joshuaeichorn.com/9OPL so your readers can know what domain they are going too before they click on the link.

Now down to the fun part installation.

Step 1, Sign up for Su.pr

Just go to the su.pr homepage and sign up using suprjosh for the invite code (The code is good for 500 sign ups).

Step 2, install and configure the PHP re-director

I’m not going go over this in detail since instructions are already available from su.pr.

Note: when installing the rewrite rule you’ll want to make sure its not catching any of your other pages, I put mine after all my custom rules and added a condition to check the given url wasn’t the name of a file or directory.

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^[a-zA-Z0-9]{4}$ supr.php?supr=$0
RewriteRule ^supr_settings.json$ supr.php?check_install=$0

If your doing your URL rewriting in PHP you might want to handle the link detection there instead, but 4 character words with no extension aren’t too common.

Step 3, Test is out

Now that you are up and running any short url created on Su.pr for your domain will be use your domain instead of su.pr for the shortening. If you ran into any problems let me know, I will see what I can do to help.

Details on how things work

• The settings on your server tell Su.pr if it has a redirector
• If the redirector is enable all new Su.pr urls for the matching domain are made with your domain
• Requests go to your server and are picked up by the rewrite rule
• The redirector on your server makes a call to the Su.pr API which returns a domain redirection url, something like http://su.pr/d/joshuaeichorn.com/abcd
• The user is redirected back to su.pr and then forwarded to the correct page on your server

The double redirection at the end lets us collect stats and do the required setup for showing the Javascript version of the Stumbleupon toolbar.

For the last couple weeks I’ve been working pretty heavily with Hbase. It decidedly has different usage patterns then Mysql, but since I don’t often get to use anything relational in Mysql because of performance reasons it hasn’t been to hard to wrap my head around. Of course having an Hbase committer a couple desks over has helped the process a lot.

So far the biggest issue has been getting access in PHP, we use thrift for that but the raw API leaves a lot to be desired. Luckily it wasn’t too big of a job to wrap things. Now I get joys of schema-less tables, atomic counters, great performance, and reliability. Now i just need to figure out how to get a small cluster for my own projects in a cost effective way.

There will be a server outage today starting around 6:30pm Arizona Time

The servers will be getting new IPs during this time so assuming we don’t run into DNS problems everything should be back up in around 2 hours.

I will twitter the status of the outage since this blog will be down too.

Services affected include:

• Webthumb
• svn.pear.php.net
• phpdoc.org
• this blog
• My other random sites

If you are a webthumb customer and have any concerns email me at jeichorn at gmail.com
Hopefully I will have an alternate endpoint up during the outage, but I just heard about the data center problems 15 minutes ago so I’m not sure if i will get it up in time.

I got to setup 3 new projects in the PEAR2 sandbox today

• Console_CommandLine – Port of PEAR1 code, php 5.2 for now
• Template_Dwoo Smarty compatible template engine, code hasn’t been imported yet, author is working on a 5.3 plan see current website for 5.2 code
• template_engine – Simple template engine, Travis’s example package for how he thinks namespaces should be used in PEAR2

Not really what I expected to be the first round of new code, but its nice to be setting up access for people. Oh and remember PEAR2 is targeted at php 5.3 but not everything is namespaced yet since not everyone wants to run snapshots of php for development.

Oh and PEAR Elections are open, 7 people ran for 7 spots so its not all that exciting, but you should vote anyway.

I’m heading off to php|tek tomorrow. I’ll be giving 2 talks I hope to see you all there. I hope my slides are complete as well .

Microsoft has released information about whats it adding to IE8. The changes for developers look pretty good.

The biggest things I see are:

• The selectors API (built selecting elements by css, there is an inprogress w3c spec for this, Webkit also implements the api in its beta)
• Better developer tools
• Lots of new AJAX stuff

AJAX wise most of the changes are implementing bug fixes and new APIs from the HTML 5 spec (DOM Storage, firing hashChanged events, XDM).

There also seems to be a new XDR API for making requests to other domains. The API is simple but I’m not sure if the security model is good enough not to cause long term problems. If anyone finds/writes a review of XDR and its security implications please let me know.

Last month I had a change to goto Microsofts WebDev07 conference. I haven’t gotten around to doing a write up but Travis did a good one if your interested in what Microsoft has to say to a bunch of PHP guys.

One of the projects I found out about while I was there was fastcgi support in IIS. If your running PHP on windows with IIS this is a pretty big deal since it gives you a stable high performance way to run PHP. I also think its important because its real engineering time by Microsoft on the on project, a real commitment to PHP.

You can grab the bits at: http://www.iis.net/php, this release is fully supported by Microsoft which is also kinda nice.

It would be interesting to see some benchmarks comparing the new fastcgi stuff against your other PHP on windows options, but its doubtful I’ll run them since I run all my production on linux.

I upgraded to WordPress 2.3.1 yesterday and do to some earlier failed upgrade all my post categories got messed up. Luckily I only have about 300 posts so re tagging everything wasn’t that painful. But word to the wise, if your running WordPress from SVN use a branch, it was following trunk in the paste that got my database messed up in the first place.

The PhpDocumentor team hopes to release the final 1.4.0 stable feature release on 7/20. If you have test-driven any of the recent pre-releases for 1.4.0 (alpha1, alpha2, RC1, RC2), please let us know any good/bad feedback you have from that experience.

Changelogs:
1.4.0RC2
Includes no PEAR items:

Includes these Sourceforge items:
- [1733938] regression: “-” not allowed anymore in converter (file)name
- [1733936] “phpdoc” does not return with error code “0″ on success
- [1733345] inline @link tag renders wrong way
- [1722096] DocBlock template example does not work

1.4.0RC1
Includes these PEAR items:
Bug #11409: PHP Notices accompany Global Never Found errors

Includes these Sourceforge items:
- [1724990] @static tag example
- [1724884] object return parameter value
- [1720764] documentation update needed
- [1672589] missing elements, bad links with HTML:Smarty:HandS
- [1634153] require/include with concatenated file name
- [1632496] {@internal}} becomes wrong on documentation
- [1124133] <code> and @include….

1.4.0a2
Includes these PEAR items:
Bug #10864: PhpDocumentor dies when parsing illegal page-level docblock
Bug #10870: Dead links generated with HTML:Smarty:PHP Bug 10871: Wrong rendering of inline {@internal}}
Bug #10910: DOM templates create incorrect trees
Doc #10914: Can’t use nested tags inside or
Doc #10972: @method does not work
Doc #11032: param type1|type2 not documented
Doc #11143: Provide a Minimal Tutorial File Setup Example

Includes no Sourceforge items.

1.4.0a1
Includes these PEAR items:
Req #10670: Don't Die on Empty Tag Descriptions
Doc #10675: Clear some errors.html items
Doc #10730: Replace Sourceforge Trackers with PEAR

Includes these Sourceforge items:
- [1647423] memory_limit: bad if-clause
- [1622538] List undocumented classes/functions
- [1602988] Added --ignore-symlinks option
- [1588942] Allow memory_limit setting in phpdoc.ini
- [1588936] Show INI Path at Startup
- [1585840] Refactored RETURNs #3
- [1574047] Refactored RETURNs #2
- [1574043] Refactored RETURNs #1
- [1543289] Apply ignore patterns to subdir of dir, instead of full path
- [1543287] Use predefined PHP env var in phpdoc script
- [1540666] patch for @property, @property-read and @property-write tags
- [1044752] List undocumented classes/functions

Tuesday I did a presentation about PEAR at my local PHP user group. Those slides are now available.

I’m hoping to improve the slides a bit and give this presentation at lots of other places. If any PHP User groups on the west coast would like me to do a presentation about PEAR drop my a line.

If you’d like to give a presentation about PEAR at your local user group also let me know and I’ll give you a tarball of the slides, you just have to agree to give me back your updates .

I’d like to thank everyone who left a comment on my last post about How you would improve PEAR. You’ll be happy to know that were already working on many of your suggestions, Greg’s post on the subject has some more details. I’ll be sharing everyones comments with the rest of the PEAR group at our meeting on the 13th, so if anyone has a suggestion they haven’t submitted yet thats your deadline.

Out of all the suggestions the one I’m most excited about is starting a mentoring program to help developers get involved with PEAR. This is someone we were already planning on doing so now its just a matter of getting the details taken care of. I’m not sure if will end up with an official sign up for a mentor or some other process but until then, if your interested in getting started as a PEAR developer and want a mentor send me an email and i’ll find a mentor for you.

Also if anyone left a comment and it didn’t show up it got caught in my spam filters. I get about 500 blog spams a day so I can’t easily check the filter for false positives. If you think this happened to you shoot me an email and post again and i’ll make sure it goes through.

You might have heard that I’m a member of the PEAR group now. Because of this I’ve been spending a lot of time thinking about what I’d like to see changed in PEAR to make it a better project. I’m slowly building up a list of ideas and at some point I’ll share it, but today I’d like to get some feedback from everyone who reads this blog.

If you were a member of the PEAR group what would you change about PEAR. Please keep in mind were not an all powerful body (Read the constitution for details), but we do have the ability to set policy, and like in any open source project we can make changes through our direct actions.

Thanks in advance for your comments.

The PEAR elections start tomorrow, if your a PEAR developer you’ll want to read people’s position statements so you can make an informed vote.

I hope these elections will be the beginning of the a rebirth of PEAR. It was a wonderful idea when it started but its been coasting for the last couple years we never did enough to embrace PHP5.

If your wondering what the whole election is about its pretty simple. A month or so again PEAR voted to accept a new constitution. This constitution, created a new position of President (Which only Greg Beaver is running for) and made the PEAR group an elected position (candidate list with position statement follows). The PEAR group’s job is to decide how PEAR should be run and too make this happen.

This election is for a term running May 1, 2007 to May 1, 2008

The following are running for the PEAR Group (Position statements are linked for their names)
Updated April 19th 2:45PM MST

• David Coallier PEAR Page
• Bertrand Mansion PEAR Page (nomination post)
• Christian Weiske PEAR Page
• Helgi Þormar PEAR Page
• Arnaud Limbourg PEAR Page (nomination post)
• Martin Jansen PEAR Page
• Paul M. Jones PEAR Page (nomination post)
• Joe Stump PEAR Page
• Joshua Eichorn PEAR Page
• Brett Bieber PEAR Page
• Stefan Neufeind PEAR Page
• Justin Patrin PEAR Page
• Mark Wiesemann PEAR Page

Some people haven’t replied with a position statement yet, so I haven’t linked to it, i’ll update this post as people’s emails make it too the list.

If I didn’t find position statements posts today I added a link to the post the person sent at nomination time.

The phpDocumentor team is just about ready to do a bug fix release. The code is sitting on the server and ready to be uploaded, we just want to get some extra testing done.

If you could install 1.3.2 do your normal documentation run, and leave a note on this post I’d be grateful. The bug fixes have all been tested but you never know what can creep in during the packaging process.

Pear download

pear upgrade http://bluga.net/projects/phpDocumentor/PhpDocumentor-1.3.2.tgz

Non-PEAR download (extract and go)

Update (2007-04-04) we made some small changes to the build, mainly making sure everything knows its version 1.3.2

Thanks to Greg’s efforts I’m now hosting a mirror of the PEAR channel. At the moment were only mirroring installer data not the entire website but its a start.

Things are still in the testing phase, but the mirror is usable, just run: pear config-set preferred_mirror us.pear.php.net

Right now we sync the mirrors every 4 hours, so it could be slightly out of date, but not enough to cause problems.

I found an interesting project, quercus that compiles PHP to Java Bytecode allowing it to run ontop of the Java Resin Application Server. It has a couple neat features, one being PHP6 compatible unicode support, the other being easy integration between Java and PHP code.

It also seems to perform well, implements a large part of the PHP api, and lets you implement anything your missing just be writing a new Java Class.

It will be interesting to see if a project like this can actually take off.

I haven’t blogged lately but I’m going to try to get with it again. Last month I finished up a big project built on top of WordPress, and then started a new job. I’m doing less PHP coding in my day job for the time being which has been a nice change.

I’ve also been doing a lot more work on phpDocumentor lately, I’ve been mainly been focused on code review since Chuck Burgess has been getting tons of patches ready. We’ve also started planing for 2.0 release, so hopefully will get the large project performance problems fixed in the next 6 months. In the shorter term was almost ready to do a bug fix release, thanks to Chuck’s work a the majority of the outstanding bug have been fixed.

On the HTML_AJAX front there is a 0.5.1 release that is almost ready to go (I fixed a bunch of bugs last week) but it still needs more testing. Because of the testing problems I’ve finally started writing some unit tests for it. I’ve been focusing on the JavaScript side, using jsunit. Hopefully i’ll have enough tests done by the end of the week to feel comfortable about releasing. The code is pretty close to 1.0 at this point, its just a matter of writing tests, finishing the manual and adding a bit of polish. Let me know if your interested in helping out. Polish and documentation aren’t the most interesting parts of Open Source development but its one of the most important parts.

I haven’t got any new AJAX work to a state where I could do a real article but I have been playing around with some new stuff. Last week I started experimenting with Dojo, I wasn’t really happy with the state of its documentation, which is really problematic because of the enormous size of its API but it does have a lot of neat features. What interested me most was its Cross Domain XMLHttpRequest support, Its based on running the XMLHttpRequest requests in a separate IFrame and then loading the results back by modifying the Fragment identifier on the parent document.

I’d also like to point out that there has been that JQuery 1.1 has been released (and a couple bug fixes releases since then). It cleans up the API bit and has some nice performance enhancements. If your using JQuery its an upgrade well worth doing.

I’ve also been working on WebThumb a bit. No big changes but I have worked out all the PayPal integration bugs. I’ve also got enough paying customers to pay for some server upgrades should be done by the end of the month. I was hoping to have a new server running month but I’ve ran into some snags. I have the new server ready to ship to the Co-lo but it seems its all maxed out on power so we can’t use the rest of the space in the rack. Hopefully things will be figured out soon. I’ll be adding at least one new feature to WebThumb this month so if you have some feature you’ve been waiting for let me know, maybe your request will be the next addition.

A couple years I got to spend some time going to the phplondon user group. Its was a great group full of knowledgeable PHP developers who love to have a good time. They have been successful at growing the group over the last couple years and just let me know that they are having there second conference. The conference is a one day event, and from what I read about last years should be a great event.

CONFERENCE DETAILS:
phplondon conference 2007
23rd February 2007 @ The Keyworth Centre, London

After the success of the 2006 php conference, phplondon is organising a second conference for anyone who has an interest in all things php. You will be able to listen to an exciting mix of topics from these diverse and thought provoking speakers.

- Cal Evens (Zend) – My First Mashup
- Simon Laws (IBM) – Web services – drop it into Apache and away you go!
- Kevlin Henney – Objects of Desire
- Rasmus Lerdorf (Yahoo!) – Fast and Rich Web Applications with PHP 5
- William (Bill) Gaver (Goldsmiths University) – Ludic Interfaces

You will also have the opportunity to network with others and share experiences both at the conference itself and chat with the speakers informally at a special event afterwards.

Early bird price is £50, or £75 on the day. To find out more and book online visit us at http://www.phpconference.co.uk

When your building fancy AJAX websites one thing that tends to happen is you end up loading amounts of JavaScript and CSS on your pages. And while browsers are smart and do a lot of client side caching you can’t get rid of that weight on your first page load.

For example on my blog I have about 60K of JavaScript and 10K of CSS. Now this isn’t horrible but when you figure images and 90K of HTML it doesn’t take long to get to my 200K total page weight.

There are a lot of various approaches for cutting down the size of JavaScript and CSS. Some of the common ones are removing whitespace and comments for JS and CSS, or using scripts which use alternate smaller syntaxes. There are a couple problems with this approach first it just doesn’t save as much space as I’d like, you can cut 60K of JavaScript down to 30K but not 10K. Second it makes debugging horrible, every JavaScript or CSS error comes from line 1 or the file.

There is a better approach and it comes from a technology in our browsers called Content Encoding. With the right headers we can send gzip’d content to the browser and it will automatically uncompress it. All modern browsers support this so its a huge win.

So now we have figure out how to compress our content, the simplest option for static files is Apache’s mod_deflate since chances are its already installed and you can enable it with a couple minutes of poking around in your httpd conf file. But it does have the disadvantage of recompressing the files for each request so it uses up some extra cpu.

mod_gzip has a similar feature set to mod_deflate with the addition of the ability to cache the compressed files but what I really want is a solution that can run on any run of the mill apache server.

To make this happen we need to compress a JavaScript file by hand and then make Apache serve it up with the correct headers.
The headers are pretty simple, Content-Type should be text/javascript and Content-Encoding should be gzip

To set the needed headers you just need to add the following rule to apache, it can go in your httpd.conf or in a .htaccess file.

<files *.js.gz>
ForceType text/javascript
Header set Content-Encoding: gzip
</files>

You can see this in action at with a small sample I made.

Now im sure your next question is what browsers support this, from my limited testing that FireFox 1.5+ and IE 6+ both work fine. From what i’ve ready anything back to version 4 of most browsers should work.

The only only downside to this approach is you have to make sure the keep your gzipped versions of files up to date and you have to update all your code to refer to the gzipped version. I don’t have a solution to the first problem, though running gzip file.js -c > file.js.gz is handy if you want to have both the uncompressed and compressed version of the file hanging around.

I thought I had a solution for the second problem using mod_rewrite. However I haven’t been able to get it working right so if you have any tips here’s my current attempt. The conditions seem to work right, but once the rediret happens the force type and content encoding header don’t seem to get set anymore.

<ifmodule mod_rewrite.c>
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_ACCEPT} >gzip
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.+).js$ $1.js.gz [QSA,L]
</ifmodule>

Note: For testing that things are working correctly you’ll want a tool to view the headers of the http response. If you want a web based tool web-sniffer seems to work well.

In my AJAX book, Understanding AJAX, I spend a lot of time talking about the various programming styles you can use with AJAX. Looking back at the chapter it seems to me there are 2 major styles of programming and large number of varitions depending on what data encodings were using, and where the bulk of our programming actually happens.

Looking at my own day to day development I tend to fall into a couple main approaches.

• AJAX page chunks (sending standard GET/POST requests, and updating part of the page by innerHTMLing in the respose)
• RPC using generated proxies
• Widgeted AJAX using RPC in the background

Now each of these approaches has its advantages and disadvantages. AJAX page chunks are easy to understand, on the PHP side im generating HTML using a standard MVC framework, and I can just turn off my outer wrapper for the chunks. The disadvantage is that my JavaScript code doesn’t have a lot of power in this approach. I’m skipping full page reloads but I still might be updating a large percentage of the page for a very small change. Its also hard to update disperate parts of a page. To solve this I sometimes find myself returning an array of HTML chunks but this quickly leads to both the PHP and JavaScript sides of the equation getting overly complicated and brittle.

AJAX Page chunks are especially nice when dealing with forms, you just serialize the form to a query string, POST it to the PHP side and return the updated form, replacing the innerHTML of the forms parent with an updated form. I consider page chunks to be my first option, they make it easy to use a single code base to support AJAX and non-AJAX operation, and most importantly are an easy concept for new developers to pick up. It also leverages current skills and keeps the amount of JavaScript down since devlopers are more likely to look towards PHP (or whatever backend lanuage you use) to solve there problems.

RPC style AJAX using generated proxies is very powerful while still being pretty simple. HTML_AJAX makes programming in this style extremly easy to implement. On the PHP side I write a class that becomes the external API to my application. HTML_AJAX covers exporting the API and encoding the data, and on the JavaScript I get a generate proxy that makes that API easy to work with.

Then its time to write a lot of JavaScript code. The biggest benefit of any RPC approach is you have data in your JavaScript not just some HTML. I tend to use an approach like this when i’m building a JavaScript application. It tends towards a more client server model and moves things away from a standard MVC approach. The biggest disadvantage to this approach is the complexity. You end up with huge amounts of JavaScript code, and much smaller amounts of PHP. This wouldn’t be horrible if JavaScript was mature and consistent on all browsers but its not. Plus you can’t trust the client so the PHP code still ends up being larger then you’d like since it has too enforce all the security and integrity rules of the application. But at the end of the day sometimes you need the power.

The last style I use is related to the more standard RPC approaches but I’ll often use it along with AJAX page chunks. With widgeted AJAX I develop a self contained set of HTML and JavaScript code that can be easily integrated into any page. This might be a autocomplete/suggest style dropdown or a live grid. I normally use RPC on the backend inside these widgets since they need actual data but what there using doesn’t really matter. The point of the widget is too make things self contained enough that they can be easily integrated with anything.

Looking at these styles I think each can have it place, but I think my decisions on what to use are more and more driven by simplicity. When looking at a problem whats the simplest solution. The more I use AJAX the more I see that its added complexity is its biggest problem. Like any other programming the simplest solution is usually the best one.

I’d like to hear your thoughts on the matter, what AJAX programming styles do you use and why?