Nearest Neighbor Image Unit
Saturday, February 6th, 2010I originally wrote this as an application using NSImage (with NSImageInterpolationNone), but decided to rewrite it as an Image Unit. So, here it is.
I originally wrote this as an application using NSImage (with NSImageInterpolationNone), but decided to rewrite it as an Image Unit. So, here it is.
The last time I released a version of IconGrabber was only a week after Valve released Half-Life 2—way back in 2004. That game wasn't even on my radar then, since I couldn't run it on my PowerPC-based Mac!
Just over five years later, I've played all of the Half-Life 2 games and love them, and IconGrabber returns with some bug fixes and support for the new bigger icon sizes introduced in Tiger and Leopard. Version 2.0.1 is available from the IconGrabber home page.
One of the ongoing debates among users of iPhone OS devices is whether an app's settings belong in the app itself, or the Settings app.
I'm of the opinion that this wouldn't even be a debate if it weren't for Apple's prescription in the iPhone HIG that every iPhone app's settings should be in the Settings app. Mac apps don't come with prefpanes for their preferences (with the exception of faceless background apps like Growl). Windows apps don't, either, that I know of. GNOME and KDE apps don't pollute Ubuntu's Control Panel.
The iPhone is the only OS I know of whose developer recommends that app developers put their application settings in the system-wide Settings app.
As we've seen several times on every platform, it's OK to break one of the local Human Interface Guidelines if and only if the violation makes the interface better.
I think this guideline is one that iPhone developers should violate flagrantly.
But there's a problem. The iPhone doesn't really have an icon for a Settings button. Most developers seem to use the
icon that one of the frameworks apparently provides, but this isn't the proper use of that icon. The
means info, not application settings.
Another choice is the gear icon for Action buttons:
But, again, we have a conflation of functions. The button in question is not an Action button; it is a Settings button. This icon is not a Settings icon. (I suspect that most people who use the Action icon use it because it doesn't have any particular association with “action”, either, other than Apple's endorsement of it for that.)
The Iconfactory, wisely, chose differently in Twitterrific. I suspect that this was largely coincidence, as the Mac version of Twitterrific came first and already had a Settings icon; for the iPhone version, the developers simply used the same icon. It works well enough:
But it's not perfect. A wrench does not say “settings”. (I offer myself as evidence: When I first saw it in the Mac version, I didn't know it was the Preferences button.) Generally, a wrench means “edit this”, as in the context of a game.
What we need is an icon that says “settings”. Ideally, this icon should either convey the notion of a changeable state value (as the previously-favored light switch [Mac OS X through Tiger] and slider [Mac OS] did), or build on an existing association with the concept of settings.
Let's go with the latter. I nominate the Settings app's icon:
Familiar enough, wouldn't you say?
That's Apple's version. Here's my button-icon (a.k.a. template) version, in the 16-px size:
I tried it out in the iPhone version of Twitterrific on my iPod touch. Before and a mock-up of after:

After I created this icon, I wondered what it would look like in the Mac version of Twitterrific.
Here's the original:
… And right away we have a problem. These buttons are already framed; my white frame will glare here.
Fortunately, that's easy to solve. With ten seconds of work, I created a frameless version. Here's what that looks like:
I think we could all get used to this icon. This wouldn't have worked at all before Apple changed the icon of System Preferences to match the iPhone Settings app, but now it can.
I don't think it's perfect. Perhaps a real icon designer (I'm just a programmer) can refine it. But I think it's a good first draft. I'm curious to hear your opinions; please post constructive feedback in the comments.
If you want to use this icon, go ahead. Here's the original Opacity document
, from which you can build all the many variations of the icon. (Click on Inspector, then Factories, then find the version you want in the list and click its Build button.)
Upon inspiration by a comment, I've just updated my rant from a couple years ago, “Photoshop sucks”, to include a list of alternatives. Topping the list, of course, is Acorn; also included are Core Image Fun House, Pixelmator, DrawIt, and Iris.
I'm very glad that there are now solutions to the problem that is Photoshop. I dislike bitching about something without a solution to offer; now I have six to offer, so that rant is now complete.
This goes out to whichever engineers at Apple fixed the Image Unit template to not suck.
Before Xcode 2.5, that template was useless. Now, it contains everything I need already set up, to the maximum extent possible.
Thank you, Apple engineers.
So you're writing an Image Unit, and it has a couple of numeric parameters. You expect that Core Image Fun House will show a slider for each of them, and it does—but no matter what you do, the slider's minimum and maximum are both 0. Furthermore, Core Image Fun House doesn't show your parameters' real display names; it simply makes them up from the parameters' KVC keys.
The problem is that you are specifying those attributes in the wrong place in your Description.plist. And yes, I know you're specifying them where the project template had them—so was I. The template has it wrong.
One of the filter attributes that Core Image recognizes is CIInputs. The value for this key is an array of dictionaries; each dictionary represents one parameter to the filter. The template has all the parameter attributes in these dictionaries. That makes sense, but it's not where Core Image looks for them.
In reality, Core Image only looks for three keys in those dictionaries:
CIAttributeNameCIAttributeClassCIAttributeDefaultAnything else, it simply ignores.
The correct place to put all those other keys (including CIAttributeSliderMin, CIAttributeSliderMax, and CIAttributeDisplayName) is in another dictionary—one for each parameter. These dictionaries go inside the CIFilterAttributes dictionary. In other words, the CIFilterAttributes dictionary should contain:
CIInputs => Aforementioned array of (now very small) dictionariesinputFoo => Dictionary fully describing the inputFoo parameter, including slider attributes and display nameinputBar => Dictionary fully describing the inputBar parameter, including slider attributes and display nameinputBaz => Dictionary fully describing the inputBaz parameter, including slider attributes and display nameFinally, an example:
<key>CIFilterAttributes</key>
<dict>
⋮
<key>CIInputs</key>
<array>
<dict>
<key>CIAttributeClass</key>
<string>CIImage</string>
<key>CIAttributeName</key>
<string>inputImage</string>
</dict>
<dict>
<key>CIAttributeClass</key>
<string>NSNumber</string>
<key>CIAttributeDefault</key>
<real>1.0</real>
<key>CIAttributeName</key>
<string>inputWhitePoint</string>
</dict>
<dict>
<key>CIAttributeClass</key>
<string>NSNumber</string>
<key>CIAttributeDefault</key>
<real>0.0</real>
<key>CIAttributeName</key>
<string>inputBlackPoint</string>
</dict>
</array>
<key>inputWhitePoint</key>
<dict>
<key>CIAttributeClass</key>
<string>NSNumber</string>
<key>CIAttributeDefault</key>
<real>1.0</real>
<key>CIAttributeDisplayName</key>
<string>White point</string>
<key>CIAttributeIdentity</key>
<real>1.0</real>
<key>CIAttributeMin</key>
<real>0.0</real>
<key>CIAttributeMax</key>
<real>1.0</real>
<key>CIAttributeName</key>
<string>inputWhitePoint</string>
<key>CIAttributeSliderMin</key>
<real>0.0</real>
<key>CIAttributeSliderMax</key>
<real>1.0</real>
<key>CIAttributeType</key>
<string>CIAttributeTypeScalar</string>
</dict>
<key>inputBlackPoint</key>
<dict>
<key>CIAttributeClass</key>
<string>NSNumber</string>
<key>CIAttributeDefault</key>
<real>0.0</real>
<key>CIAttributeDisplayName</key>
<string>Black point</string>
<key>CIAttributeIdentity</key>
<real>0.0</real>
<key>CIAttributeMin</key>
<real>0.0</real>
<key>CIAttributeMax</key>
<real>1.0</real>
<key>CIAttributeName</key>
<string>inputBlackPoint</string>
<key>CIAttributeSliderMin</key>
<real>0.0</real>
<key>CIAttributeSliderMax</key>
<real>1.0</real>
<key>CIAttributeType</key>
<string>CIAttributeTypeScalar</string>
</dict>
</dict>
You can see how the descriptions under CIInputs are as terse as possible; everything besides the absolute necessities is specified in the outer dictionaries.
Updated 2008-04-17 to clarify the marked-up screenshot of the Color Matrix view. If you've seen this post before, check out the before and after.
So, let's say you want to convert an image to a mask.
→

This is easy to do with the Color Matrix filter in Core Image.
If you've ever looked at the Color Matrix filter out of curiosity, you were probably frightened by its imposing array of 20 text fields:

Don't worry. The fields are actually very simple, though not explained in the UI:

(The documentation for the Color Matrix filter actually does explain it, but I like my explanation better for not using fancy math terms like “dot product”.)
So with this tool, our task is redefined like so:
You can generalize this to the extraction of other channels. Let's say you want to make a mask of the blue channel:
To demonstrate this, here's a red-blue gradient (shown in Acorn to visualize the gradient image's own transparency):

If we extract the blue channel, as shown above, we get this:

Note how the red parts of the gradient are black, because we extracted the blue channel, and there was little to no blue there.
Likewise, if we extract the red channel, we get this:

In this case, the converse of the blue-channel mask.
(By the way, in case you're wondering: No, I don't know what caused the white pixels along the edge. It could be a Lineform bug, or a Core Image bug, or a graphics-card bug. I didn't keep the original Lineform file for the source image, stupidly, but in case you'd like to test it on your own machine, I re-created it. Here's a PDF of the replica; you can use image to convert it to PNG. I can confirm that I saw similar results with this image to the results with the image I used for this post.)
You can even mix up the colors of an image. Suppose we want to reverse that gradient:

The red-to-blue gradient is now a blue-to-red gradient.
Well, mainly, so you can create mask images. Several filters require these, such as the Blend with Mask filter in the Stylize category. The Color Matrix filter makes this easy, although you still have to save the mask image somewhere.
It's even easier in Opacity, where you can create a Color Matrix filter layer, configure it using the Layer Inspector, then hide it by clicking its eye icon. This way, the filter layer won't show up in the rendered document (or in any of its build products), but you can still use its result as the mask to another filter layer.
As you may have read on wootest's weblog, Like Thought Software released its new image editor, Opacity, today.
Before I go any further, here's full disclosure: The developer invited me to beta-test the app, and I did. He also gave me a free license for this purpose (the app normally costs $89 USD). Also, I have some code in the app, because it uses IconFamily, which I contributed a patch to a long time ago.
OK, that's everything. Now, to borrow from wootest's disclaimer on the same topic:
Don’t confuse this as simple tit-for-tat back-scratching, though. Had I … had no involvement whatsoever, the application would still have been every bit as brilliant, and I would have come out just as strongly in favor of it.
I love this app.
Opacity is an image editor designed to enable app developers to create multiple-resolution and any-resolution graphics easily. It's built for that specific purpose, and the Opacity website even says so. This app really is not intended for anything other than user-interface graphics.
Key points:
Opacity has several important features over past editors:
Opacity is not perfect. Some things don't quite work like you would expect: for example, vector objects do automatically appear in every resolution, but pixels that you draw or paste don't automatically get mirrored to the other resolutions; instead, Opacity waits for your explicit say-so (the Clone Current Layer's Pixels to Other Resolutions command). Opacity also still has a couple of major bugs: Flip Horizontal, for example, takes way too long in one document that I created. Personally, I didn't expect it to go final this early, and I recommend that you wait until at least 1.0.1.
But those are dark linings in a silver cloud. Once all the major bugs are fixed, I believe that this app is how you will create your application's custom toolbar and button images for the modern resolution-independent world.
→ 
UPDATE 2008-01-02: Ahruman commented that you can just use NSNetwork in IconGrabber. No need to go through all these steps and fake one.
If you've ever needed a high-resolution version of the Network icon for anything, you may have noticed that Mac OS X does not ship with one. When you select the Network and Copy it, then create a new document from the clipboard in Preview or Acorn, the largest size available is 128-px.
Fortunately, the .Mac icon is available in 512-px, and you can easily change it into the Network icon.
You will, of course, need Leopard (for no other version of Mac OS X has 512-px icons).
The easiest way to get the .Mac image is IconGrabber. Enter the name “NSDotMac”, then click Draw, then set the size to 512×512, then save. (Note: On an Intel Mac, you'll need to build from source, because the pre-built version for PowerPCs doesn't run on Intel for some reason.)
pngout \ %~/Projects/@otherpeoplesprojects/growl/trunk/Core/Resources(0) > NotifyOSX.growlStyle/Contents/Resources/sidetitle.png In: NotifyOSX.growlStyle/Contents/Resources/sidetitle.png In: 29644 bytes Out: NotifyOSX.growlStyle/Contents/Resources/sidetitle.png Out: 527 bytes Chg: -29117 bytes ( 1% of original)
Slightly late because I had to devise a way to determine whether a GIF file is interlaced. (I settled on GifBuilder, in case you're curious.) This ties in with the next two bugs; I'll blog both at once next week.
This bug is NSImageInterlaced documented as working on half of known interlaceable types. It was filed on 2007-05-12 at 00:27 PDT.
This bug is NSFrameRectWithWidth uses the current color, not the stroke color. It was filed on 2007-04-27 at 16:17 PDT.
(more...)Today, I scanned in one of my old drawings: a study of five-pointed stars that I made when I was trying to figure out how to draw a proper star (this was at the time of me working on Keynote Bingo MWSF2007 Edition, and a derivative of the same star is used in TuneTagger).
The odd thing is, after I corrected the image using Preview's Black Point and Aperture controls (no relation to the photo-management program), the image weighed about two-fifths as much:
du -b Five-pointed\ star\ study* %~/Pictures(0) 1403443 Five-pointed star study-adjusted levels.png 3346498 Five-pointed star study.png
(These sizes are after pngout, but even if I re-correct the original image and save it elsewhere, it comes out 1790244 bytes long.)
Go figure.
I'll follow this up with a tutorial called “PostScript for Cocoa programmers”, but today brings my list of reasons why you should care in the first place.
(more...)Not too long ago, I was at the bank and decided to take this photo of a couple of magazines sitting next to each other. As you can see, I edited out the bank's address.
I did this using Lineform. The problem is, Lineform is a vector app, so it doesn't keep any EXIF data from the original image (most of the time, that would not make sense). In my situation, I did want to keep the EXIF info, but there's no way to make Lineform do that.
So I wrote a command-line tool to bring EXIF properties over from one image to another image. I call this image exif-confer. Enjoy.
Silly me, trying to use a device with the Mac drivers that come with the device. Turns out it works just fine with the built-in Mac OS X drivers, either via PTP (whatever that is), or as a mass-storage device. In fact, Image Capture works the same either way.
With the HP drivers, a program called “HPCamera_PTP” would crash whenever I plugged in the camera, whether I did this in Image Capture or iPhoto. I found that switching the camera to mass-storage mode (“Disk Drive” in the USB Configuration menu) worked around that problem nicely, and Image Capture (and iPhoto) even work transparently in this mode.
Later, I was tinkering with Image Capture in some way (I forget why) and noticed that it has its own PTP driver. This gave me an idea, and having long ago uninstalled the HP uselessware, I switched the camera back to PTP mode (“Digital Camera” in the USB Configuration menu) and plugged it back in. Huzzah! It worked exactly as it did in mass-storage mode.
Kudos to Apple for making it do the Right Thing either way. Antikudos to HP for making non-functional drivers.
I also got a new scanner yesterday, a CanoScan LiDE 600F. Unfortunately, it doesn't work without drivers. Fortunately, its drivers work. (Both devices let me use Image Capture without touching any of the apps that come with them, which I consider mandatory given the nearly-consistent asstasticity of the UIs of such apps in general.)
A few weeks ago, I installed Adobe Reader to view a particular PDF, and noticed something interesting in its Preferences:

“Wow”, I thought, “I wonder how it knows that.” So I went looking through the Quartz Display Services documentation, and found it.
The function is CGDisplayScreenSize. It returns a struct CGSize containing the number of millimeters in each dimension of the physical size of the screen. Convert to inches and divide the number of pixels by it, and you've got DPI.
Not all displays support EDID (which is what the docs for CGDisplayScreenSize say it uses); if yours doesn't, CGDisplayScreenSize will return CGSizeZero. Watch for this; failure to account for this possibility will lead to division-by-zero errors.
Here's an app to demonstrate this technique:

ShowAllResolutions will show one of these windows on each display on your computer, and it should update if your display configuration changes (e.g. you change resolution or plug/unplug a display). If CGDisplayScreenSize comes back with CGZeroSize, ShowAllResolutions will state its resolution as 0 dpi both ways.
The practical usage of this is for things like Adobe Reader and Preview (note: Preview doesn't do this), and their photographic equivalents. If you're writing an image editor of any kind, you should consider using the screen resolution to correct the magnification factor so that a 8.5×11″ image takes up exactly 8.5″ across (and 11″ down, if possible).
“Ah,”, you say, “but what about Resolution Independence?”.
The theory of Resolution Independence is that in some future version of Mac OS X (possibly Leopard), the OS will automatically set the UI scale factor so that the interface objects will be some fixed number of (meters|inches) in size, rather than some absolute number of pixels. So in my case, it would set the UI scale factor to roughly 98/72, or about 1+⅓.
This is a great idea, but it screws up the Adobe Reader theory of automatic magnification. With its setting that asks you what resolution your display is, it inherently assumes that your virtual display is 72 dpi—that is, that your UI is not scaled. Multiplying by 98/72 is not appropriate when the entire UI has already been multiplied by this same factor; you would essentially be doing the multiplication twice (the OS does it once, and then you do it again).
The solution to that is in the bottom half of that window. While I was working on ShowAllResolutions, I noticed that NSScreen also has a means to ascertain the screen's resolution: [[[myScreen deviceDescription] objectForKey:NSDeviceResolution] sizeValue]. It's not the same as the Quartz Display Services function, as you can see; it seemingly returns { 72, 72 } constantly.
Except it doesn't.
In fact, the size that it returns is premultiplied by the UI scale factor; if you set your scale factor to 2 in Quartz Debug and launch ShowAllResolutions, you'll see that NSScreen now returns { 144, 144 }.
The Resolution-Independent version of Mac OS X will probably use CGDisplayScreenSize to set the scale factor automatically, so that on that version of Mac OS X, NSScreen will probably return { 98.52, 98.52 }, { 96.33, 96.33 }, or { 98.52, 96.33 } for me. At that point, dividing the resolution you derived from CGDisplayScreenSize by the resolution you got from NSScreen will be a no-op, and the PDF view will not be doubly-magnified after all. It will be magnified by 133+⅓% by the UI scale factor, and then magnified again by 100% (CGDisplayScreenSize divided by NSDeviceResolution) by the app.
Obviously, that's assuming that the app actually uses NSScreen to get the virtual resolution, or corrects for HIGetScaleFactor() itself. Adobe Reader doesn't do that, unfortunately, so it suffers the double-multiplication problem.
So, the summary:
{ 72.0f, 72.0f }. For example, in my case, you would scale by { 98.52, 96.33 } / { 72.0, 72.0 } (that is, the x-axis by 98.52/72 and the y-axis by 96.33/72). The correct screen to ask for its resolution is generally [[self window] screen] (where self is a kind of NSView).-[NSStatusBar thickness], which return a number of pixels rather than points (which is inconvenient in, say, your status item's content view).Can you imagine an Image Unit that didn't actually use Core Image?
I just wrote one.
Well, OK, so I did use CIFilter and CIImage — you can't get away without those. But I did not use a CIKernel. That's right: This simple filter does its work without a kernel.
For the uninitiated, a kernel is what QuartzCore compiles to either a pixel shader or a series of vector (AltiVec or SSE) instructions. All Image Units (as far as I know) use one — not only because it's faster than any other way, but because that's all you see in the documentation.
But I was curious. Could an Image Unit be written that didn't use a kernel? I saw nothing to prevent it, and indeed, it does work just fine.
The image unit that I wrote simply scales the image by a multiplier, using AppKit. I call it the AppKit-scaling Image Unit. Feel free to try it out or peek at the source code; my usual BSD license applies.
Obviously, this Image Unit shouldn't require a Core Image-capable GPU.
So I'm looking at making some screencasts in iShowU. I noticed that, thanks to QuickTime, it supports rather a lot of codecs:

Dizzying, isn't it?
So I asked the Goog about some of the codecs, and also more generically about screencast codecs. H.264 is a popular choice, but I found no comparison to other codecs nor any discussion of what any of the codecs is particularly good at.
I also had a small amount of previous experience with the difference between codecs. I've watched two of rentzsch's screencasts: “Introduction to Core Data” and “Embedded Cocoa Frameworks”. The former is 1024×768 using H.264; the latter is 800×600 using Sorenson 3. “Introduction to Core Data” looked flawless; “Embedded Cocoa Frameworks”, OTOH, had noticeable artifacts. I wondered at first what the difference was, and after I found out, whether there wasn't a better codec with which to implement screencasts.
Part of the problem is that there really isn't a codec that's been well-optimized for screencasting. Video codecs have historically been optimized for, well, video — that is, video shot with a camera. Screen images usually don't change much except to follow mouse movements; a compressor specifically optimized for recording the screen should make a big difference in screencast file sizes. Are you listening, Apple? (Or Ambrosia?)
Anyway. In order to find the perfect (so far) screencast codec, I recorded a brief screencast using iShowU in demo mode using the None (no-compression) codec, and used QTAmateur by Michael Ash to export it to all the other codecs.
The results? Well, you'll just have to read the screencast codec showdown for yourself. ☺
You may remember Daniel Jalkut's blog post about resolution independence, with its demonstration of the superiority in quality of vector graphics over raster graphics (using an EPS file that I created for him ☺).
The problem with using an EPS file, as I pointed out to him when I emailed him the file originally, is that EPS files are very slow on OS X. The current (Tiger) implementation of NSEPSImageRep uses Quartz's CGPSConverter to convert the EPS data to PDF data, then instantiates an NSPDFImageRep (kept in a private ivar) that it has do all the real work.
That PostScript-to-PDF conversion is the bottleneck. You don't want that multiple-second delay (yes, really) at runtime. Better to take it before runtime. At the time, I suggested that he use pstopdf to convert to a PDF file, then put the PDF file into his Xcode project to be copied into the app bundle.
But what if you decide later to change the PostScript file? You need to remember to run pstopdf again. EPS files are just PostScript source code, so why not treat them like any other source code file? Or, a better question: How?
The answer is quite simple: Add a custom rule to your target in Xcode that runs pstopdf over EPS files. Then, you add the EPS file — not the PDF file — to the project, and Xcode will exhibit normal build-only-when-necessary behavior on the EPS file and copy the generated PDF file for you. And of course, you can have as many EPS files as you want — they're source code files just like any others.
