How ClickToFlash works
Speaking of everybody’s favorite WebKit plug-in, here’s how it works. This should help you understand how it fails on some sites, and maybe aid you in contributing to its development.
First off, it is a WebKit plug-in; it’s written in Cocoa and uses WebKit’s own plug-in API. It does not use the Netscape plug-in API.
A WebKit plug-in declares a list of MIME media types in its Info.plist bundle. It does this by way of a dictionary of dictionaries:
<key>WebPluginMIMETypes</key>
<dict>
<key>application/x-shockwave-flash</key>
<dict>
<key>WebPluginTypeDescription</key>
<string>ClickToFlash</string>
</dict>
</dict>
WebKit only loads a WebKit plug-in when it first encounters some content that it needs the plug-in for. It uses these Info.plist dictionaries to know which plug-in it needs to load. So, in the case of ClickToFlash, it only loads ClickToFlash when it encounters something that is of type application/x-shockwave-flash.
Here’s the brilliant part. Adobe’s Flash Player plug-in actually declares* two MIME media types: application/x-shockwave-flash and application/futuresplash.
You will notice that ClickToFlash only declares one of these.
As it turns out, everybody only uses application/x-shockwave-flash. ClickToFlash exploits this.
When you click on the ClickToFlash view, it modifies the object or embed element that the view represents, changing its type attribute to the other type—the one ClickToFlash doesn’t declare; the one no webpages actually use. WebKit notices this change and looks again for a plug-in to handle the movie. This time, it comes up with only one handler: the real Flash Player plug-in.
There are several reasons why a site may not work with ClickToFlash. I suspect one reason is that they try to interact with the movie via JavaScript; ClickToFlash doesn’t export a script object and wouldn’t be able to communicate with the real Flash Player anyway. The script finds itself talking to a wall, and breakage happens.
So now that you know how ClickToFlash works, maybe you can help fix its bugs?
* Flash Player doesn’t declare its MIME types in its Info.plist; it declares them in a resource file, in ‘STR#’ resource 128. Thanks to WebKit developer Mark Rowe for reminding me to look for a resource. ↶
January 30th, 2009 at 11:54:54
When looking for a suitable plugin for a give MIME type, why does WebKit load ClickToFlash instead of Flash Player? Is the namespace collision resolved via “first to load”(e.g. ZZZClickToFlash wouldn’t work) or is preference given to Info.plist entries before STR# resources? Anyone know?
January 30th, 2009 at 12:40:14
ClickToFlash is loaded as WebKit prefers to load WebKit-style plug-ins over Netscape-style plug-ins if both claim a MIME type.
January 30th, 2009 at 12:52:42
I think this is great stuff, but I’m concerned that sites depending on Flash-based ad revenue (note to such sites: use static, non-CPU-hog banners and nobody gets hurt) will change something to defeat this. Any speculation what such counter-measures might be?
January 30th, 2009 at 12:53:42
Excuse me if I’m being naïve, but would this break if Adobe modified the mime type strings to only be one type?
I sure hope they don’t get around it, but they do seem intent on ramming flash down our necks via the web.
January 30th, 2009 at 14:51:27
Cute. I wrote a Flash-blocker plugin for my own use with a different architecture. It loaded a private captive copy of the real Flash plugin, but only fed plugin events to it after the first click event. It didn’t work as well as ClickToFlash because Flash content would still run without drawing, so the CPU and network costs were just as high.
January 30th, 2009 at 17:28:10
This makes me wonder how feasible it might be to detect certain kinds of Flash videos and redirect them to the QuickTime plugin. On YouTube it’s possible to work out the URL to an (often higher-quality) MP4 from a video’s embed URL, and if you have the Perian codec package installed you can play FLV (after finding the actual video URL in the embed/object tag of many players) via QuickTime… either of which might be more desirable than the quirky player UIs and turn-the-fans-up-to-11 CPU inefficiency of handing them off to the Flash plugin.
January 30th, 2009 at 19:07:54
Duncan: There’s one pretty obvious countermeasure. I’ve added a bit of text in the middle of the post; it should clear it up for you.
drew: I wouldn’t worry too much about it. It might reduce advertiser usage of Flash, but not nearly as much as Google has and not as much as FlashBlock, either; moreover, it may actually promote other use of Flash because it saves you from having to turn Flash off entirely in order to keep the ads from annoying you.
January 30th, 2009 at 22:28:35
I don’t mind ads as much as I mind non-standard content on a web page. If doubleclick and other ad purveyors detect that you don’t have flash, they’ll offer gifs and jpegs instead; I’m OK with these since I know the bulk of the free web requires them in order to stay in business but theses flash ads tend to be way too obtrusive. Unfortunately, I don’t think a plug-in can be written that intercepts flash and makes the ad flash detection logic think flash isn’t installed so that images are served but the flash content is fetched when the image is clicked.
January 30th, 2009 at 22:47:50
Mark Malone: In your case, you may prefer to turn plug-ins off, uninstall Flash, or switch to a brower (such as OmniWeb) that lets you turn specific plug-ins on and off.
February 9th, 2009 at 07:16:04
Rick: Jonathan Rentzsch’s current ClickToFlash source (what will be 1.4) has a “Load H.264” contextual menu item that does that for YouTube videos. As you suggested, it switches the movie over to QuickTime. Thank Troy Gaul for adding it.