Webthumb updates
Webthumb got off the ground yesterday without too many problems. It ran into problems right out of the gate since a large number of sites don’t seem to want to finish loading correctly. I was able to fix this by adding an extra timeout handler to the browser code. If we stop making progress loading a site for 5 seconds we just give up and snapshot what is currently loaded.
I also updated the server config, Webthumb now supports flash (version 7), and has a lot more fonts installed so sites that need non-latin characters should be working correctly.
My current feature goals are:
Better Url’s, something like /thumbs/url/time
5 minute cache on thumb generation (if anyone else has requested the same site in the last 5 minutes just return those images)
A way to get a thumb of a site without going through the queue, if its already been generated
A way to search for current thumbs
User accounts with:
Snap at time
Snap recurring
I’m also looking at prototying and API for access, though it may require paying a fee to use.
If you have any feature suggestions or input on an API leave a comment.
Feature suggestions (besides better filenames):
- selectable size(s) of capture area (800×600,1024×768,…,custom)
- selectable size(s) of thumb
- possibility to download all the generated thumbs from a url as one zip
- option to turn vertical and/or horizontal autoscroll on, so that a full page can be captured even if it exceeds the client window of the capturing browser
Joshua Eichorn’s Blog: Webthumb updates…
…
No suggestions, just wanted to say it seems like an awesome project.
Definately do the API thing, I’d pay
That there is pretty damn cool, handled some pretty heavy JavaScript of mine just fine.
Wow thanks for the cool services….works great on drupal sites
nice work ^^
by any chance, is there any API snippet code that would not crequire url and php5 to use this service ?
curl is not installed on most webservers (given the multiple security issues), and php4 is still more popular than php5 ..
be well
I haven’t written any but it should be really easy to take the code from Hasin, and update it too use HTTP_Request. I just use a little JavaScript tester for making sure things work so I don’t have any PHP code that talks to the api.
WTF?
Notice: Undefined index: referrer in /home/htdocs/jeichorn/bluga.net/webthumb/index.php on line 32
Warning: Cannot modify header information – headers already sent by (output started at /home/htdocs/jeichorn/bluga.net/webthumb/index.php:32) in /home/htdocs/jeichorn/bluga.net/webthumb/index.php on line 39
I wasn’t handling a missing referrer correctly, im guessing your run a privacy proxy or something. Anyhow it should be fixed now.
Hi! I love the site. I would be willing to pay for a service which let me download large thumbnails quickly for use (for example) in a search engine like Snap.com. I don’t mind if they are cached for up to a month, so long as you’re hosting them!
Some page’s still not finish loading proprely like this one : http://www2.canoe.com/index.html
MarcAndre, for the time being there isn’t much i can do for sites like that. Something about there layout makes they take an extremly long time to render even after all the content is downloaded.
I’ll be looking at better load detection techniques in the future but there doesn’t seem to be any easy solutions.
How does one purchase credits for webthumbs, I suspect I’ll need 5000 per month with my project
Hi.I wanted to sign up for a webthumbs account but nothing happened when I filled out the registration form.
mccormicky: i broke something, things should be working now try again
Hey quick question, what browser is the snapshot generated from? My site seemes to display ok in ie and firefox on mac and pc but the snapshot being generated is showing an alignment issue that I can’t seem to locate.
Thanks
Its a mozilla browser running on linux. I think its the equivalent of firefox 1.0.8 The fonts might also be slightly different.
I was just testing a few URLs. There’s a problem in your code when I attempt to thumbnail :
http://www.latimes.com/news/nationworld/world/la-fg-haditha22dec22,1,5819163.story?track=crosspromo&coll=la-headlines-world&ctrack=1&cset=true
Wow this is awesome.
Is there a time-delay or are the images generated LIVE on-the-fly?
I mean if I generate a thumbnail, does the image represent the site as it was when I clicked or it from some time ago. If it is not live, is there an indication of how reent the image is?
Thanks,
They are generated on the fly, thumbnails normally take 5-20 seconds to generate depending how busy the server is.
There is one exception if your using the recently added easythumb features images are cached for at least 24 hours.
Hi,
We launch a service called Menthe-fresh (http://www.menthe-fresh.fr) and we use your service. We would like to mention that there is a little bug on your price list.
You mention on your home page that it is $20 for 15,000 credits.
“Registering gives you API access and free 250 credits a month, additional credits can be purchased at any time, and cost $20 for 15,000 credits.”
I am a register user and I see that it is :
$20 – 5,000 WebThumb Credits
$50 – 15,000 WebThumb Credits
How much should I pay ?
Loopion:
Sorry about that the list on the purchase page is the correct amount:
$20 – 5,000 WebThumb Credits
$50 – 15,000 WebThumb Credits
For the benefit of any other users, I’ve written a Ruby API for Webthumb. You can get the source at http://snippets.dzone.com/posts/show/3621
Seems like there is a problem with flash widgets.
http://www.fantasyfestivals.com/festivals/15
Garth: I don’t see a problem http://bluga.net/webthumb/thumbnail/www.fantasyfestivals.com_festivals_15/2007-03-23@09:41:42
Excellent resource. Struggling to get the API running without curl, but it definitely looks like it is worth percevering with. Has anyone had any joy converting Hasin’s code for non-curl environments?
I tried a page that displays a quicktime movie from Darwin Streaming Server. Actually, there is a poster movie (still) displayed first. The thumbnail shows the “Get Plug In” dialog.
Can you provide a list of content types and plugins that are supported?
Hey matbennett,
Have you tried using PEAR Http_Request?
Bill:
I’m only running the flash, and PDF plugins right now. Other then that I support everything firefox 1.5 does.
I’m looking at adding support for direct thumbnailing of movies but that will be a related service.
Coming up with a good thumbnailer for movies is very tough. When we posted a movie to Youtube, we found they grabbed a frame from the middle (I think). But the middle frame wasn’t a good representative, so I had to add black frames to the end of the movie until the middle was a decent image.
The page I was trying to thumb was http://www.raisethebar.com/play.taf?_uid1=2, which is a streamed, autostart quicktime movie (mpeg4). I was mistaken that we had a poster movie for it, as this page dynamically embeds the movie player. With autorstart, I don’t know what you would have grabbed, even if you had an mp4 plugin installed, it would be very timing dependent.
The same is true if I had asked you to thumb the movie directly http://www.raisethebar.com/movies/Buzz_Report.mov (rather than embedded in a page). I think this is a problem that requires some eyeballs to solve.
Bill: I was actually planning on doing a thumbnail set on the direct movie thumbnail service. Plugin wise i may be able to support more media types for people who want the video in a page to look right, but that depends on me getting the new server setup which i’ve been dragging my feet on.
Is the source code available for webthumb?
This is really great, is there anyway to get a complete full size image?
I’d love to use this with a commercial project I’m slated to start on in a few weeks in which we’d need to purchase a lot of keys. It would be a highly trafficked site with a lot of requests to your capture engine. I don’t think we’d have time to setup any of the open source alternatives on our own servers.
The only issue is I would need to actually get the full screensize back, not a scaled thumbnail.
Is there anyway to do this? It seems like on the server side it always scales the images.
In any event, cool project!
-dan
Dmitri:
The PHP API wrapper is open source
An older version of the custom mozilla browser is open source, i have a private branch of that code at this point.
The api etc isn’t open source
daniel:
There is an undocumented feature that lets you get the full image, it makes requests cost 2 credits instead of 1.
You just add <fullthumb>1 to the request like:
<webthumb>
<apikey>yourkeyhere</apikey>
<request>
<fullthumb>1</fullthumb>
<url>webthumb.bluga.net</url>
</request>
</webthumb>
You thumbnail service (which is great!) does not capture YouTube videos, could this issue be fixed? I would like to create thumbnails of flash video (if you have not already guessed it).
Did you disable your flash plugin? You mention in your post that you have added flash support. I just started using this service (which rocks!) and I remember that I was able to see the flash objects, but now I cannot (maybe I am just going crazy).
Marques
marques:
For videos you would really need a specific version just for them to get the results you want. I’ve looked into creating one but thats a matter of finding the time to get it up and running.
The Flash Plugin is working again. The service leaks shared memory and when it hits its limits the plugin doesn’t work right anymore.
Thanks Joshua for fixing the flash plugin, actually I already have a solution for creating thumbnails for wmv/mov/flv files directly, if that is what the other posters are looking for. Contact me and I can give you details.
Thanks!
Sorry to bug you but some urls do not work.
http://bluga.net/webthumb/thumbnail/www.youtube.com_watch%3Fv=_fpDTGLQ3Bs/2007-06-08@18:40:19
I looked in the pageprint file and found a possible solution to the errors that I am getting when using your services.
// quoted out existing code and kept as a reference/backup
//$command = “$dir/PagePrint-bin -i $procUniq $cookieHack$args 2>&1″;
// My cheat,
$command = “$dir/PagePrint-bin -i $procUniq $cookieHack -s ‘$argv[2]‘ $argv[3] 2>&1″
I’ve noticed that your computer doesnt have the font “Trebuchet MS” installed. Is this because the server is on Unix/Linux? I’m not sure if you could add it, but it would be cool if you could.
JiveBay:
Its a Linux box, so it has some different fonts installed from the default windows set. I’ll see what options i have for adding some Microsoft fonts.
I got the Microsoft Fonts from the WINE (windows emulator) page once for a linux box.
Here is a useful post on Digg (the comments are good too)
http://digg.com/linux_unix/Adding_Windows_Fonts_in_Linux
Here are the 6 new Vista fonts also:
http://blogable.net/archives/2005/07/31/windows-vista-fonts/
If you could add all the standard Windows web fonts that would be cool.
Hi Joshua,
we love your service and use it for our worldwide database of scuba diving shops.
A nice feature would be to be able to capture the full page, if it is longer than one screen. Something like autoscrolling.
Any chances to get that into the service?
Matthias
Matthias:
Adding autoscrolling for full page capturing sounds like an interesting feature. But i’m not sure when i’ll have time to work on it.
An extremely useful service though it appears to have stopped working properly. Are you letting it go or do you want the facility mirrored elsewhere?
Things are working fine, I just moved the server and missed redirecting http://www.bluga.net since its not a url I ever use. The redirect is in place now.
Matthias: I didn’t add autoscrolling but I am supporting snapshots with the web browser set to 2048 pixels high.
Just tried it and it works perfectly, thank you. An excellent facility.
Hello, first of all thank you for providing such a great application. I have been using it extensively to provide image previews of ppl’s weekly schedules so that they can post it on their newsfeed for facebook.
I had purchased 15,000 credits to handle the number of requests, but after the month had finished it seems that they do not recharge like the standard 250. I was wondering if this is standard, and if so will you be offering purchase options for rechargeable points. paying 100-150 every month adds up, and I don’t know if I want to continue doing that for my ever-growing user base.
Thanks a lot, I hope to be able to continue using your services in the future.
Matt:
The extra credits are one time credits. If you see yourself using more then 15-20K credits a month contact me josh@bluga.net and we can discuss bulk rates.