There and Back Again

Webthumb Rendering Engine Released

Webthumb uses a custom browser thats embeds the mozilla rendering engine to take its website screenshots.

We call that custom browser pageprint, and its svn repo is now available. The browser was originally written to generate PDF’s for Clearhealth so most of my recent development has centered around that. If anyone has tips on packaging or making the code work with newer embedding sources instead of mozilla 1.7.x i’d be most appreciative.

I know for certain the code won’t work with seamonkey 1.0.3

Its also pretty raw, but the binary in svn should work with centos and debian without any problems.

34 thoughts on “Webthumb Rendering Engine Released

  1. boris

    I’ve done it without any c/c++ and php and i’m using the newest mozilla with xvfb.

    Xvfb :2 -screen 0 1024×768×24 -fbdir /tmp &
    DISPLAY=:2.0 mozilla &
    DISPLAY=:2.0 mozilla -remote “openURL(javascript:window.resizeTo(1024, 768);)”
    DISPLAY=:2.0 mozilla -remote “openURL(http://www.example.com)”
    sleep 5
    xwd -display :2.0 -root -out image.xwd
    xwdtopnm image.xwd > image.pnm
    pnmscale -xysize 200 150 image.pnm > imagesmall.pnm
    pnmtojpeg imagesmall.pnm > image.jpg

  2. Cal Evans

    Joshua,

    This is truly a cool tool. Do you have an API for it? If not, do you have plans for an API?

    I can see in the blogging world how this would be useful but I’d like to write a script that I can just hand it a URL and possibly a description, have it use your code to gen a thumbnail and then automatically upload it to Flickr. (or in my case, the DevZone server.)

    =C=

  3. Joshua Eichorn Post author

    Boris: an approach like that works, but has browser chrome and doesn’t have an easy way to detect broken loading. Plus it doesn’t handle the case i’ve done the development for which is unattended printing.

    Cal:
    There are plans for an api, im just not sure when i’ll get things done.

    Mike:
    GPL

    Epaulin:
    I’m glad you got it working, though i have binaries checked into svn so you shouldn’t have needed to compile at all.

  4. Pingback: Web App Development

  5. Pingback: WebRatings Team’s Blog » Blog Archive » WebThumb 关注ing.

  6. Vivek Misra

    Hi!

    Is there any way to install this thing on leased space . I have purchased some space and would like this to work there .

    Is there any alternative ? could you please share it : what exactly i have to instruct my web hosting provider ?

    (sorry to use ternary operator there )

    kewl Tool Joshua

    regards

    Vivek

  7. Joshua Eichorn Post author

    Vivek: your best bet is too used the soon to be released API. This isn’t the sort of code you’ll want to run if your not the only one on the server.

  8. Hans Rakers

    Great tool, thanks for your efforts!

    I have adapted PagePrint succesfully to work with seamonkey 1.0.3. If you are interested in my changes, contact me :)

  9. egor

    Hans Rakers, can you sen sources on my e-mail?
    c0mat0z[at]gmail.com
    regards, egor

  10. Joshua Eichorn Post author

    Hans:
    Thanks for the patches, getting them checked in is on my todo list. But i’ll be traveling again this week so im not sure when i’ll get it done.

  11. baoeni

    I am a newbie of linux and when I tried with the boris code:
    Xvfb :2 -screen 0 1024×768×24 -fbdir /tmp & &

    I got an error:
    _XSERVTransSocketINETCreateListener: …SocketCreateListener() failed
    _XSERVTransMakeAllCOTSServerListeners: server already running

    Fatal server error:
    Cannot establish any listening sockets – Make sure an X server isn’t already running

    What does that mean and what should I do to make it running?

  12. Joshua Eichorn Post author

    Either you have an Xserver already running or you have something else listening on the Xvfb port (i’ve seen ssh pick ports in the X range before). Changing the screen # to something else will change the port.

  13. baoeni

    I am curious about how did you know when mozilla has finished loading the page?

  14. Joshua Eichorn Post author

    Baoeni:
    I have a custom browser so I can capture an event when the network load is complete. But thats not perfect since there is still rendering time etc so i just wait for a couple seconds.

  15. Lost

    Hi
    This is probably a total noob question, but after running

    Xvfb :10 -screen 0 800x600x16 &

    and then trying to run PagePrint, I get

    Xvfb: client 1 rejected from local host
    Xlib: connection to “:10.0″ refused by server

    After some googling, I tried

    xhost +

    but that didn’t seem to help. I also tried running Xvfb on a different display (:6), and that was the same result.

    Any ideas?

    Thanks!

  16. Joshua Eichorn Post author

    Lost: Are you running Xvfb and pageprint as the same user, and do you have another xserver running on the machine.

    Also what distro and X version are you running

  17. Lost

    I was running them as the same user, but didn’t realize that I couldn’t have another xserver running…

    OK, now that I’ve changed inittab to run mode 3 and rebooted, I get past the “client 1 rejected” problem only to get another:

    Document Load Started
    Document Load Started
    Document Load Stopped
    Attempting to Print
    Failed to get nsIWebBrowserPrintClosing Mozilla

    I’m trying this on a box with FC5, if it makes a difference.

    Thanks!

  18. Joshua Eichorn Post author

    Lost: you could be able to work with another Xserver running but im not sure what you need to do to get the permissions worked out.

    I haven’t run things on fc5, does that come with mozilla 1.7.8 or seamonkey 1.0.5

  19. Lost

    If I echo out the $mozDir variable in PagePrint, I get /usr/lib/mozilla-1.7.12, if that helps.

  20. Joshua Eichorn Post author

    Lost: I’m using 1.7.13 and its working, though i haven’t tried the newest svn where i’ve only done builds with seamonkey

  21. Charles Duffy

    How do I actually check out that repository? The WebSVN instance browses just fine, but I don’t see a URL which can actually be plugged into subversion itself.

  22. marques

    Josh, how do you execute pageprint from a php script? I can create thumbnails from the command line, but if I try to exec(./pageprint), my server gets in a endless loop forcing me to reboot.

  23. Anugrah Widya

    dear joshua,

    i would like to run your webthumb in my server,
    but i always get segfault when running you binary build PagePrint-bin,

    [root@molinux webthumb]# ./PagePrint –snapshot http://www.petitmerci.local petitmerci
    string(115) “/var/www/html/widi/webthumb/PagePrint-bin -i PagePrint46b88fd9d0924 –snapshot http://www.petitmerci.local petitmerci 2>&1″
    Url: http://www.petitmerci.local, Output: petitmerci
    Starting PagePrint
    Profile path is: /root/.PagePrint
    Loading URL: http://www.petitmerci.local

    Configuration:
    minWidth: 640
    maxWidth: 1024
    minHeight: 480
    maxHeight: 768
    smallWidth: 80
    smallHeight: 60
    mediumWidth: 160
    mediumHeight: 120
    medium2Width: 320
    medium2Height: 240
    largeWidth: 640
    largeHeight: 480
    browserWidth: 1024
    browserHeight: 768
    outputFullSnap: 1

    sh: line 1: 12127 Segmentation fault /var/www/html/widi/webthumb/PagePrint-bin -i PagePrint46b88fd9d0924 –snapshot http://www.petitmerci.local petitmerci 2>&1

    the line number of segfault always change.

    and for info, currently i’m using fc6 as testserver (where webthumb will put)

    hope you can help to solve this

  24. George P.

    Hey Joshua,

    I tried the link above to download from svn, but it didn’t work. Could you please let me know were I can download this from? Also, would pageprint work in a server environment with no gnome?

    I really appreciate your efforts,
    George

  25. Joshua Eichorn Post author

    George: I don’t run that svn so there isn’t a lot i can do.

    I believe that svn has been moved to http://svn.clear-health.net/svn/pageprint/

    Also note i’m not longer maintainer that code (I’ve forked things locally) but i’ll answer questions if i can.

    Environment wise you can runs things on a server but you need an minimal X environment installed (depending on how firefox/mozilla/seamonkey is built some gnome stuff too). You just match things up with Xvfb and you should be good.

  26. Marques

    Has anyone made an attempt to updated pageprint so that it uses the firefox rendering engine instead of seamonkey? I am in process of moving to a new server that has Centos 5 install and their are no RPMs available for seamonkey. Joshua I would imagine that your server/OS is at EOL and wanted to know if you were planning on tackling this issue?

  27. Joshua Eichorn Post author

    I have a forked version of this code base (greatly refactored) that I use for webthumb. It runs on xulrunner and thats the route i recommend for anyone that wants to update things. But i’m not going to be releasing my fork since it doesn’t handling printing and I rather not put webthumb out of business.