Singletons in Cocoa: Doing them wrong

 

2009-06-17 15:19:51 UTC

A singleton is a simple concept: It's an object that you can only have one of. Being that simple, you may think it's hard to get wrong.

If you do, I welcome you to the wonderful world of programming.

The Cocoa Fundamentals Guide is necessary reading for all Cocoa programmers. Even so, nobody's perfect, and the CFG's authors are no exception.

On Twitter last night, a mistake of theirs came to light when someone named Nick tweeted about how hard it is to make a singleton in Cocoa. Nick cited an example from the CFG; Chris Hanson responded that “that example does WAY too much work”, and Bill Bumgarner added that the example is “ridiculous”. They're both right.

Doing it wrong.

The example, in order to “ensure singleton behavior”, overrides four basic Foundation methods:

  • retain
  • release
  • autorelease
  • retainCount

In the override implementations, retain, release, and autorelease are all no-ops. release does exactly nothing; the other two do nothing but return self. The override implementation of retainCount unconditionally returns UINT_MAX.

Before I proceed any further, I'll give you a caution about singletons. You generally do not need to make your object a singleton at all. Even if your application has a single-window interface, with only one of each controller, you probably don't need to enforce that. Just don't create more than one. Then, if you do create more than one, you have a bug.

Let's digress for a moment. Imagine if you allowed it. Imagine that you allow, say, your app delegate to create more than one of your root controller object. Your app delegate will then proceed to set up both twin objects, and they both respond to such things as notifications.

Bad, right? Now let's say you “fix” this by making your root controller a singleton.

Your app delegate is still trying to create two root controllers. It's still trying to set up two root controllers, and as a result of the two set-up messages, the controller is signed up twice for notifications.

But now you really only have one root controller. Your app delegate creates the first one, then gets it again when it tries to create the second. All according to plan so far. Then your app delegate sets up the first object—and then it sets it up again, again thinking that it's talking to a second object. Even worse, because the object is now signed up twice for notifications (once per set-up message), every notification response happens twice.

You now have only one root controller, but you didn't fix the bug, which wasn't in the controller at all, but in the app delegate. To fix the bug, you must fix the app delegate; you don't need a singleton for this at all.

OK, digression over. Singletons are bad. Avoid them. If you have a lot of them, rip most of them out. (Dave Dribin has bookmarked a lot of other good cases against singletons, and BJ Homer points out that they aren't all bad. More on BJ's post later.)

Back to the singleton at hand.

First, let's look at retainCount. A comment explains that UINT_MAX “denotes an object that cannot be released”, implying that the framework itself considers UINT_MAX a special value.

This is actually correct, although I originally thought (and wrote, in an earlier version of this post) that it was bogus. The documentation for retainCount explicitly says that objects whose release method does nothing should return UINT_MAX.

Next on the hit list: autorelease. This is just mostly pointless. autorelease means nothing more than “send this a release message later”. If release does nothing, then autorelease becomes “do nothing later”. All this override does is move the nothing up to now. A performance optimization, perhaps, but I'd say it's premature.

And now we come to the real villains: retain and release.

First off, you shouldn't retain a singleton anyway. I can't imagine why you would do this, except incidentally, by making it a timer target or something.

But even if you do think of a reason to retain a singleton, you still need to balance retain and release. If you retain without releasing afterward, you are under-releasing the object. If you release without retaining previously, you are over-releasing the object. These two statements have no exceptions.

If you break retain and release, then you're able to over-retain or over-release (or even both!) the object with impunity. Your app no longer crashes, but you didn't really fix the problem; you're just getting away with it. You're still trying to over-retain and/or over-release an object.

The Cocoa Fundamentals Guide's primary audience is new Cocoa developers. Those who have never used a retain-counting system before may very well over-release the singleton, and Apple's example singleton implementation hides that bug from them. That's bad; every Cocoa programmer should know how to recognize an over-release crash, and breaking retain and release denies the reader an opportunity to learn that. I've filed a documentation bug report about the example.

Also, a rebuttal

BJ Homer responds at a different angle to last night's flurry of tweets.

First, though, we need to get a definition straight. A singleton is a class of which there should only be one instance in any given process. There are actually very few singleton classes in the Cocoa framework including NSDocumentController and NSFontManager. You cannot create more than one of these objects; …

This is true. A singleton object is an object that your app can only have one of. There are a couple of different ways to do this:

… if you try to call [[NSDocumentController alloc] init], you'll get back the exact same object as you do when you call [NSDocument sharedDocumentController], no matter how many times you call alloc and init. NSApplication is arguably a singleton as well; you can alloc another one, but you'll get an assertion failure when you call init.

Another way is to simply implement the sharedFramistan method, and leave allocWithZone: and init alone. I suspect that this is common, since it's the easiest way, but BJ is right that it isn't a true singleton, since it allows multiple instances.

Where BJ goes wrong is in his defense of the retain and release overrides:

Consider the following example:

MyFloozit *floozit1 = [[MyFloozit alloc] init];
[floozit1 doSomething];
[floozit1 release];

MyFloozit *floozit2 = [[MyFloozit alloc] init]; // MyFloozit is a singleton, so this should be the same object as floozit1
[floozit2 doSomething]; // CRASH HERE
[floozit2 release];

We'll leave aside the problem that you probably should be using sharedFloozit to obtain the singleton instance of MyFloozit.

When floozit1 is set, a new MyFloozit is allocated, and a static MyFloozit pointer is set. When floozit1 is released, that static pointer is still pointing to the old instance. As a result, when we try to set floozit2 (or when anyone else tries to call [MyFloozit sharedFloozit]), we get back a pointer to that same instance. The one that has been dealloc-ed.

BJ is missing something else the CFG says, which he even quoted later on:

Situations could arise where you want a singleton instance (created and controlled by the class factory method) but also have the ability to create other instances as needed through allocation and initialization. …

(Emphasis added by me.)

A singleton object owns itself. As such, it retains itself. As such, it should never die, because it always has at least one owner—and that's without breaking retain and release. If it does die, it's because something over-released it; later, something that was using it will crash, which lets you know that you have a bug. This is a good thing, because now you can fix the bug.

Each of the release messages in BJ's example balances an alloc message above it. That alloc message may actually return an existing object, but we're expecting to own a new object. Therefore, the singleton's allocWithZone: should implicitly retain the existing object.

There is no good reason to override retain and release. Don't do it. This also goes for autorelease. And, since you never override release, you also never need to override retainCount.

Doing it right

So, having thoroughly dismantled the Apple documentation's poor implementation of a singleton, let's look at the correct way to do it.

Let's go through the requirements:

  • The One True Instance is the only instance. (If you deliberately allow multiple instances, I call this a “multipleton”. I'll leave that as an exercise for the reader, and concentrate on the true singleton here.)
  • There is a sharedFramistan method. It tests whether the One True Instance exists; if not, it creates the object and remembers it in file-scope static storage. Then it returns the One True Instance.
  • We'll allow going through alloc and init, and return the same instance. We'll do this in allocWithZone:, as Apple did. We'll also need to make sure init doesn't do its work twice on the same object.
@interface PRHEmptySingleton: NSObject
{
    BOOL hasInited;
}
@end
static PRHEmptySingleton *sharedInstance = nil;

@implementation PRHEmptySingleton

+ (void) initialize {
    if (!sharedInstance)
        sharedInstance = [[self alloc] init];
}

+ (id) sharedFramistan {
    //Already set by +initialize.
    return sharedInstance;
}

+ (id) allocWithZone:(NSZone *)zone {
    //Usually already set by +initialize.
    if (sharedInstance) {
        //The caller expects to receive a new object, so implicitly retain it to balance out the caller's eventual release message.
        return [sharedInstance retain];
    } else {
        //When not already set, +initialize is our caller—it's creating the shared instance. Let this go through.
        return [super allocWithZone:zone];
    }
}

- (id) init {
    if (!hasInited) {
        if ((self = [super init])) {
            //Initialize the instance here.

            hasInited = YES;
        }
    }

    return self;
}

@end

Now, your tests:

  1. sharedFramistan always returns the same object.
  2. alloc/init always produce the same object (which, itself, is the same object as sharedFramistan).
  3. alloc/init will not return an object confused by multiple init messages.
  4. Over-releasing causes a crash.
  5. Keeping alloc/allocWithZone:/retain balanced with release never causes a crash.
  6. If [super init] returns a different object, alloc/init won't break.

Apple's example fails test 4 (and 6, for that matter). BJ doesn't show an implementation for his example, but according to his own description of its behavior, it fails test 5. My example above passes all six tests.

Take-aways

Hiding bugs is bad. Even worse is giving code that can hide a bug to new Cocoa programmers who could really use practice in detecting and fixing that kind of bug.

If you really need to implement a singleton, there is a right way to do it. The way currently shown in the Cocoa Fundamentals Guide isn't it.

Don't change the behavior of retain, release, retainCount, or autorelease. Ever.

*whoosh*

 

2009-06-03 23:17:00 UTC

In March 2007, Merlin Mann interviewed musician John Vanderslice. I've downloaded four of his songs since I started downloading lots of free music, and every time, I've remembered that I had first heard his name from that episode of The Merlin Show.

Just now, I went back and watched it again. At one point, they talked about MP3 blogs (Vanderslice used to run one), and Vanderslice mentioned one specifically: Largehearted Boy.

That discussion, and that specific mention, went right past me the first time I watched this interview. Now, more than two years later, Largehearted Boy is probably the single biggest contributor to my music library.

Amazing what you miss sometimes, isn't it?

(Incidentally, Vanderslice has some free songs and albums available for download on his website.)

UPDATE 23:59: Don't miss part 2. Even more interesting than the first part, especially looking back from 2009.

Targus Chill Mat follow-up

 

2009-05-24 11:45:53 UTC

I wrote previously about the Targus Chill Mat. Two Chill Mat generations later, it's time for an update.

When I wrote the previous post, I had only the first-generation Chill Mat, and Targus had just introduced the second generation. I finally bought one a few months ago.

It does seem more effective at cooling than the first-gen, but the price I paid for that is that it's loud—so loud that I would intentionally put it aside and go back to the first-gen for my monthly trips to CocoaHeads Lake Forest.

Last week, visiting Walmart for the first time in a long time (they remodeled it! it sucks less now!), I noticed that there is now a third generation. I picked it up for $20, partly out of curiosity regarding just where they stashed the cable, since I couldn't see it from outside the package.

For those of you not familiar with the Chill Mats, they're powered by USB. One end looks like the end of a DC power cable (as from a wall-wart), and the other end is a USB A connector.

The first and second generations had a problem where bouncing around inside your laptop bag with the DC connector plugged in would cause the connector to become loose, and eventually stop making reliable contact. The first-generation also had an inline power switch, which was even easier to break; they knocked that off with the second-gen, so let us not speak of it again.

The third-generation adds a nifty feature: Set into the underside of the mat is a hollow cleat, with a clip on each side of it. You wrap the cable around the cleat and hold it in place by snapping it into the clips, and you put the USB connector inside one end of the cleat and the DC connector inside the other end.

This cleat feature should help relieve the fragility issues that plagued the DC connector on previous Chill Mats, because now I have a better place to store the cable than plugged into the jack.

The third-generation is also much better about noise (to the point that I'll feel comfortable bringing it to CocoaHeads), but the trade-off is that it cools much less effectively than the second-gen. Even so, I think it's good enough. (In case you're wondering, it is a little louder than the first-gen.)

All told, this is the best Chill Mat yet, and it will be my Chill Mat for all uses from now on.

UPDATE 2009-05-24: When I went to add links to this post, I noticed the Chill Mat for Mac on their website. I haven't seen this in any store. If you find or order one, I'm curious to hear how well it works (you can set iStat Menus to show your MBP's external temperature) and how loud it is.

Popularization

 

2009-05-11 20:55:32 UTC

Here's something I found interesting.

If you're on Twitter, you might have noticed a trending tag today, #TMItweets. People have been deliberately posting tweets with Too Much Information, tagged with #TMItweets.

The interesting part is who started it.

If you go all the way back on the Twitter search for the tag, you'll find that the earliest tweet in the search engine's short memory is this one from Alex Fayle, 20 days ago. I don't think he's who started the current trend.

The next few tweets are. They are, in order:

  1. Tai (her tweet)
  2. Dora Bianchi (her tweet)
  3. Hannelore Ellicott-Chatham (her tweet)
  4. Raven Pritchard (her tweet)
  5. Pintsize* (his tweet)
  6. Penelope Gaines (her first tweet with the tag; her second)
  7. Faye Whitaker (her tweet)

If you read Questionable Content, you recognize those names. These are the official QC character accounts, they each have thousands of followers, and they're all written by Jeph Jacques.

So, basically, this trend started among a closed circle of fictional characters, and migrated into the real world from there within minutes**. I can't think of a medium where this could have happened before (without including it in the primary medium—in this case, in the comic strip).

Like I said, I found it interesting.


* Not linking directly to Pintsize's account page because the links he posts are almost all NSFW. Consider yourself warned.

** Pintsize posted his #TMItweets entry at 5:05:46 UTC, and the first subsequent non-QC entry came at 5:07:53 UTC. I used the Twitter API show-status method through curl to obtain these timestamps.

Scale

 

2009-04-29 10:25:27 UTC

264 doesn't sound much bigger than 232, does it?

264 is 232 × 232, but this relation is hard to put into perspective. Even saying that 264 is more than four billion times as big as 232 doesn't adequately convey how much larger it really is.

Let me do that for you now.

I wrote a program that counts from 0 to 232. It takes about 11 seconds to run:

% ./count-up
2009-04-29 09:52:55.387 count-up[57932:10b] Time to count up to 4294967295: 11.313282

Now, as I said, 264 is 232 × 232, so the time to count to 264 is likewise the time to count to 232 (11.313282 seconds) multiplied by 232. That works out to:

  • About 48,590,176,200 seconds
  • About 13,497,271 hours
  • About 562,386 days
  • About 1540 years

Simply by doubling the exponent, we increase the time it would take my late-2006 four-core Mac Pro to count up to the number from 11 seconds to a millenium and a half.

Symbolicator 1.0.1

 

2009-04-24 23:51:58 UTC

It's two-thirds faster, it works on more crash logs, and thanks to Augie Fackler, it's now available from the Python Package Index. This means that you can just do this:

sudo easy_install symbolicator

If [you run a custom build of Python and] you don't want to install setuptools, you can always get the Symbolicator from its webpage.

Oh, and there will be a 1.0.2 to fix some issues. At the moment, I'm going back to working on Growl things, so I won't start on those for awhile; if you want to beat me to them, fork the project on Bitbucket, commit your fixes, and send me a pull request. (Make sure you co-ordinate your efforts on the ticket! I don't want to have to choose from two independently-developed fixes.)

UPDATE 2009-04-25: Clarified above that installing setuptools is only necessary if you have installed Python yourself. Thanks to Augie for pointing this out in his comment.

Put this in your ~/.hgrc file:

[merge-tools]
filemerge.executable=opendiff
filemerge.args=$other $local -ancestor $base -merge $output
filemerge.gui=True

(Based on the original description of and Matt Mackall's comment on a Mercurial bug about merging.)

New tool: The Symbolicator

 

2009-04-19 15:45:32 UTC

The most significant behind-the-scenes change in Growl 1.1.5 is that we now build using the DWARF-with-dSYM debug symbol format.

Now, we distribute a fully-stripped executable for every release—even betas—and we keep the dSYM bundles on hand separately. The idea here is that we can use the dSYM bundles to obtain symbolic information (function name, filename, and line number) for the bare addresses in users' crash logs.

Unfortunately, there are no good tools for symbolicating Mac OS X crash logs. If this were an iPhone app, we could just drag the crash logs into the Xcode Organizer window, but Xcode doesn't do this for Mac crash logs.

I tried all the other tools as well, and every one of them had one of two problems:

  1. Didn't work at all
  2. Needed the dSYM bundle and main bundle to be next to each other on disk

#2 is not a deal-breaker, but it is a hassle.

So I wrote my own symbolication tool.

The Symbolicator is a Python script that:

  1. Reads in a crash log.
  2. Finds the necessary dSYM bundles using Spotlight. (This means that you can put the dSYM bundles anywhere you want, and as long as Spotlight can find them, the Symbolicator will be able to use them.)
  3. Uses dwarfdump to extract the relevant symbol information.
  4. Replaces the address offsets with the symbol information (just like CrashReporter does when it has debug symbols to work with).
  5. Writes the symbolicated log to stdout.

Use it like this:

% symbolicator < unsymbolicated.crash > symbolicated.crash

Or use ThisService to make a service out of it.

If you want to see it in action right now, download Growl 1.1.5b1 and the corresponding dSYM bundles from the Growl beta page. Make Growl crash (killall -TRAP works well), then unpack the bundles and use the Symbolicator on the crash log. (If you unpack the bundles first, CrashReporter will symbolicate the log before you even get to it. Handy, unless you're trying to test the Symbolicator. ☺)

To make this work on your own app, follow the instructions in the aforementioned ADC article, then make sure you archive the dSYM bundles for every release, including betas. On Growl, I added code to our Release Makefile for this.

Note: If your app is closed-source, you should not put your dSYM bundles on your website, since the debug symbols are arguably trade secrets (information about your source code). Keep them locally, perhaps on a flash-memory drive. Disclaimer: IANAL.

Manpage Monday: PlistBuddy(8)

 

2009-04-13 11:35:44 UTC

PlistBuddy(8) is a command-line tool for editing property-list files, with a deeper reach than plutil defaults:

Entries consist of property key names delimited by colons. Array items are specified by a zero-based integer index. Examples:

  :CFBundleShortVersionString
  :CFBundleDocumentTypes:2:CFBundleTypeExtensions

Jonathan “Wolf” Rentzsch, in his del.icio.us bookmark for the manpage, says that PlistBuddy is “always at /usr/libexec/PlistBuddy on modern systems”.

Don’t pass objects as context pointers

 

2009-04-03 09:27:24 UTC

Cocoa has several functions and methods that run a panel (usually as a sheet) asynchronously. These functions and methods take a target object, a selector, and a context pointer, and when they finish, they send a message to the target with that selector, passing that context pointer.

A common practice is to pass an object in the context pointer. You retain the object before you call the run-panel function or method, then release it in your callback. For example:

    NSBeginAlertSheet(/*title*/ NSLocalizedStringFromTableInBundle(@"Update Available", nil, bundle, @""),
                      ⋮
                      /*modalDelegate*/ self,
                      /*didEndSelector*/ NULL,
                      /*didDismissSelector*/ @selector(downloadSelector:returnCode:contextInfo:),
                      /*contextInfo*/ (void *)downloadURL,
                      …);

In this example, downloadURL is a local variable, and the object in it was retained above, or maybe not autoreleased in the first place. The downloadSelector:returnCode:contextInfo: method will cast the context pointer back to an object type, probably do something with it, then release it.

This has recently become a visible problem because of the Clang Static Analyzer. The analyzer doesn't know that NSBeginAlertSheet will retain the object, so it flags this call as a leak. This is the most-often-cited example of the checker's “false positives”: things that it flags as bugs but that aren't really bugs.

But is it really not a bug? Is it really appropriate to use an object as a context pointer?

Let's think about what we're doing here:

  1. The panel doesn't know the context pointer is really an object, so it doesn't retain it. To compensate for this, we're retaining an object on behalf of another object (the panel). This alone should make you feel dirty.

  2. The panel doesn't know the context pointer is really an object, so it won't release the object, either. This is particularly a problem if something releases the panel (quite possible in the case of, say, an NSAlert instance). Then it really is a leak.

  3. If we get released (and, hopefully, take the panel with us or at least cancel it), then we leak the object, because we are only expecting to have to release it (or even to have access to it!) in the callback.

Passing objects through context pointers is an anti-pattern. I hope the Clang Static Analyzer developers never remove this warning, because this is not a false bug—it is a real bug that we should all fix in our programs.

Solutions

That object needs to go in an instance variable somewhere.

The lightweight solution is to create an instance variable on the object that is running the sheet. You store the object in the instance variable when you run the sheet (and leave the context pointer NULL), then retrieve the object from the instance variable in your callback. You'll still need to retain it when putting it in, and release it both in your callback and in dealloc—those aspects don't change.

I can hear you saying that this is a waste of an instance variable. What are they, rationed? There's nothing to waste; instance variables are cheap. And leaking objects is far more of a waste of memory than adding an instance variable.

But this isn't the only solution.

Another way is to create your own subclass of the panel class (NSOpenPanel, NSSavePanel, or NSAlert), and give the subclass a property you can use instead of the context pointer. You could make it abstract (making the property similar to NSMenuItem's representedObject, nothing more), or make it specific to your application. The former approach is simpler, whereas the latter gives you a base on which to build a more-customized alert if you later decide you want one. It's up to you.

The advantage of this solution is that, since you've formalized the instance variable as part of a class's API (and since the instance variable is no longer a bit player but the central part of a class), it'll be harder for you to forget to manage your ownership of it correctly. Memory problems solved.

One way or another, though, you need to put that object into an instance variable somewhere to avoid leaking memory, feeling dirty, and having the Clang Static Analyzer chew you out.

(And yes, all this applies with garbage collection as well. See the section “GC Gotchas” in episode 36 of Late Night Cocoa. However, I disagree with Andre Pang's recommendation in that episode of CFRetain and CFRelease; the right solution is not to dip into Core Foundation, but to avoid the context pointer altogether and use an instance variable instead.)

Fraser Speirs has a post about configuring your Git repository to vet commits with the Clang Static Analyzer.

The idea of pre-commit hooks is that you get to run a script before the commit happens. Depending on the result code of the script, the commit will either proceed or be aborted.

I wrote a wrapper around the scan-build tool, so that I could run the analyzer by hand with my preferred options at any time: …

The --status-bugs flag is the trick here: it makes scan-build return a non-zero status code if it detects bugs. That’s what we want with Git pre-commit hooks: a non-zero status indicates a possible bug, and that causes the Git commit to be aborted.

Mercurial, of course, has the same feature. The hg book has instructions; I'll show you how I set up my repository to do this.

First, I created a shell script named RunClang.zsh:

#!/bin/zsh -f
~/bin/checker-latest/scan-build \
    -checker-cfref -warn-objc-methodsigs -warn-objc-missing-dealloc -warn-objc-unused-ivars \
    --status-bugs -o checker.out \
    xcodebuild -configuration $1

Next, I added my precommit hook to the repository's .hg/hgrc file:

[hooks]
precommit.RunClang = ~/bin/RunClang.zsh Development

Here's what an example session looks like:

% echo 'Testing precommit testing hook' >> test.txt
% hg ci -m 'Testing precommit testing hook'
[churn churn churn]
** BUILD SUCCEEDED **
scan-build: 17 bugs found.
scan-build: Run 'scan-view [snip]/growl-boredzo-precommit-test/checker.out/2009-04-03-2' to examine bug reports.
abort: precommit.RunClang hook exited with status 1

(255)% hg log --limit=1
changeset:   4188:b208862a586d
tag:         tip
user:        Peter Hosey
date:        Fri Mar 13 05:40:09 2009 -0700
summary:     Fix encoding of the Norwegian Growl-WithInstaller strings file.

17 bugs—mostly leaks. Glad I didn't commit this test file!

So now you know how to have Mercurial block you from committing if the clang checker can find bugs. This should also work for your unit tests. And if you have 100% test coverage (lucky!), you can combine them: have scan-build build your test-bundle target. Then, the hook will prevent the commit if the checker can find bugs or any tests fail.

I don't think I'll actually use this set-up, though.

First, in order to find bugs, you need to build your entire main product. Any significantly large program is going to take a long time to build and analyze—Growl, for example, takes about one-and-a-quarter minutes for a clean build. Even committing to a Subversion repository over dial-up was quicker.

More significantly, precommit hooks like this interfere with patch queues. The mq extension implements patches as mutable commits, so any qnew or qrefresh will run the hook. It would be useful on qfinish, but it's just annoying on qnew and especially qrefresh, as the all-too-frequent builds thwart rapid iteration.

So, if you use patch queues, this won't work for you. But, if you don't, then this should work as well in Mercurial as it does in Git.

Adding Growl support to Mercurial

 

2009-04-01 07:32:08 UTC

Add this to your ~/.hgrc file:

[hooks]
changegroup.growl = ((echo "$HG_URL" | grep -vF 'file:' > /dev/null) && growlnotify -n Mercurial 'Pull successful' -m "Pulled at least $(hg log --template='\n' -r$HG_NODE:tip | wc -l | sed -e 's/ //g') changesets from $HG_URL") || true
outgoing.growl = (test "x$HG_URL" '!=' 'x' && growlnotify -n Mercurial 'Push successful' -m "Pushed to $HG_URL") || true
It's not perfect: Both notifications share the same notification name. As long as that isn't a problem, this works fine.

Free song follow-up

 

2009-03-31 11:28:41 UTC

In the first three months of this year, I've downloaded 2927 free songs. That's not including the recent label samplers from the Amazon MP3 Store, nor the thousand-song influx that is the SXSW 2009 torrents.

From this, I estimate that I will download over 11,000 songs in this year alone.

My library right now, including everything I've ever bought, is 10,626 songs. Subtracting the 2927 mentioned above, I have 7699 songs that I didn't get this year, or didn't get for free. This means that this year's free music will have multiplied the size of my library by about 250%.

I say this not to brag, but because my mind is blown. I'm going to more than double my library this year, for free. Legally. Without pirating a single song.

Wow.

There's no reason for anyone to pirate music anymore. Music DRM is dead, so the “moral” argument is gone, and you can get as much music as you can listen to without having to spend one red cent. If you have a specific song in mind, you usually can buy it from iTunes or Amazon—and if you don't have the money to do that, why are you pirating music instead of looking for a job?

I do worry that this could backfire on artists. I barely have enough time to listen to everything I get for free, so I never listen to the rest of the songs on the album, which means I never buy the album. If enough people did what I do, the music economy would collapse.

You might say “why not just donate money to the artists?”. The main reason is, again, my limited time. Getting, tagging, and listening to all this free music takes up more spare time than I'd like already; now you're proposing that I spend more time just to spend some money. I don't feel guilty enough.

Two other reasons are that I don't like donating money in exchange for nothing (the song was free), nor paying for things I already have. I'd rather buy the album, because then, I'm buying something I don't already have.

Radio Paradise works well here. I hear songs that I like and must buy to have, and I buy them. This way actually makes money for artists.

But Radio Paradise isn't perfect. The main problem, yet again, is that time issue: I no longer have much time at my computer when I'm not listening to fresh free music, and when such time does arrive, I usually use it to listen to what I already have (a rare break from the endless stream of new stuff). Radio Paradise loses badly in my schedule.

I take comfort in the possibility that I'm an outlier: the only one actually gathering as many of these songs as I do, while everyone else is content to only tap a few sources (e.g., iTunes + Amazon) and make up the rest with purchases. The artists get their due compensation, and I get my free music.

Still, I can't shake the feeling that there must be some better way.

Here's something an artist could try: Set up a combination player and online store on your website. Let me listen to the entire album for free. Let me download any, say, two songs for free. If I want more than that, I have to pony up for the whole thing.

Another way would be something more like iTunes and Amazon, but with a much longer preview—let's say half the song. I don't know about you, but I find 30 seconds useless. With half-song previews, I could get a good sense of the song and whether I want it in my library, but it wouldn't be worth just downloading the preview and adding that, since it's only half the song.

The problem with both these solutions is that they, too, compete for time. I'm not sure I'd find it worth it for one artist. Perhaps a record label or independent online music store (like Insound) would be willing to try it.

The best solution I can think of would combine one of the above solutions with a streaming internet-radio player. I could open a browser window (or SSB) upon the player, and leave it running in the background. If I hear something I like, I could bookmark it, so that I could come back to it later to listen to it again, listen to the album (the whole thing, straight through), buy the song, buy the album, recommend the song on Twitter, or dismiss it.

What do you think? Am I inadvertently screwing over artists in these ideas? Should I stop looking the gift horses in the mouth and just take my free music? Are there better solutions than what I've suggested?

UPDATE 12:07 PDT: Just found this by random encounter: Amie Street, an experimental new music store. Its twist is that every song starts out “free or very cheap”, and goes up in price as more people buy it. Talk about demand-driven. I haven't tried it, though—if any of you have, please speak up.

Manpage Monday: copyfile(3)

 

2009-03-30 06:51:49 UTC

copyfile(3) is an API for copying and moving files within the file-system:

DESCRIPTION

These functions are used to copy a file's data and/or metadata. (Metadata consists of permissions, extended attributes, access control lists, and so forth.)

The copyfile() function can copy the named from file to the named to file; the fcopyfile() function does the same, but using the file descriptors of already-opened files.

The copyfile() and fcopyfile() functions can also have their behavior modified by the following flags:

COPYFILE_CHECK

Return a bitmask (corresponding to the flags argument) indicating which contents would be copied; no data are actually copied. (E.g., if flags was set to COPYFILE_CHECK|COPYFILE_METADATA, and the from file had extended attributes but no ACLs, the return value would be COPYFILE_XATTR.)

COPYFILE_PACK

Serialize the from file. The to file is an AppleDouble-format file.

COPYFILE_UNPACK

Unserialize the from file. The from file is an AppleDouble-format file; the to file will have the extended attributes, ACLs, resource fork, and FinderInfo data from the to file, regardless of the flags argument passed in.

File Manager also has APIs for copying files, in both asynchronous and synchronous flavors. Those APIs don't provide as much control over management of metadata, but they do offer asynchronous operation, whereas the copyfile(3) APIs appear to be synchronous.

And, of course, I should mention NSWorkspace operations, which you use with the performFileOperation:source:destination:files:tag: method. Unlike the other two, this API has been around since 10.0. On the other hand, like copyfile(3), it's synchronous only.

Storage

 

2009-03-27 10:00:21 UTC

The Apple ProFile hard drive held 5 or 10 megabytes in 11.15×43.89×22.38cm = 10952.17893 cubic centimeters.

A microSD card is 1.5×1.1×0.1cm = 0.165 cubic centimeters and can hold up to 16 gigabytes.

If you stacked up enough microSD cards to fill (as closely as possible*) the volume of a ProFile, you would have 29×20×111=64380 cards totaling 1.03008 petabytes (1030.08 terabytes)—206,016,000 times the capacity of the hard drive you'd displaced (assuming the 5 MB version).

Also, the ProFile cost $3499 in 1981 (about $8194 in 2008 dollars), whereas a 16 GB microSD card costs about $70 as of March 2009. If you somehow bought enough 5 MB ProFiles to make up the difference in capacity (3200 of them), at the original MSRP, you would spend $11,196,800 (about $26,222,014 in 2008 dollars).

(I used Robert Sahr's inflation conversion-factor tables to convert to 1981 dollars to 2008 dollars.)


* It's possible that you could fill out the space a bit more by standing some cards on edge, but I don't feel like doing that much multiplication.

Safari 4 beta and GrowlMail

 

2009-02-24 12:00:58 UTC

The problem

The WebKit framework that comes with Safari 4 enforces a restriction on calling WebKit from a secondary thread.

GrowlMail accesses message contents on a secondary thread because they may not be in yet. Accessing message contents can result in a call to WebKit. That results in a crash with the new WebKit.

The fix (section added 2009-04-19, updated 2009-06-22 for the 1.1.5 release)

Download Growl 1.1.5 and install GrowlMail 1.1.5.
(Make sure you install GrowlMail specifically! The Growl Installer package does not include GrowlMail or any of the other Extras.)

The workaround

UPDATE 2009-06-22: You don't need to do this anymore. Install GrowlMail 1.1.5 instead.

If you're fast or can easily turn off your internet connection first, go to GrowlMail's Preferences and set GrowlMail to summary mode. (You might also try putting Mail into Offline mode immediately after launching it until you've made this change.)

  1. Go to Mail's Preferences.
  2. Click on the chevron button at the far right end of the toolbar.
  3. Click on GrowlMail.
  4. Choose the second of the three radio buttons: “Show a summary of received emails”.

You can then go back into online mode/turn your internet connection back on/rest easy.

The other way to do it is to run this command in a terminal window:

defaults write com.apple.mail GMSummaryMode -int 2

Ordinarily, I'd tell you to quit Mail first, but since Mail will unexpectedly quit the first time tries to Growl about a new mail message, I don't think I need to bother. ☺

The fix

We're not sure about that yet.

The problem is that when Mail adds a new message to your library, it may not have fully downloaded it yet. (You can see this yourself sometimes when your internet connection is slow or under heavy load: You'll click on a message and see a “Loading” screen in the preview pane.)

When that happens, if we try to get the message body on the main thread, it blocks the UI until the body arrives. So we do it on a secondary thread instead. That's now a problem.

The ideal fix is that we find a way to determine whether the body has come in yet. If it hasn't, we could set a timer for a few seconds, then check again then and post a notification with “body not loaded yet” if it's still not in.

Another fix I would consider is simply killing the feature that shows the message body in the notification. I'm sure a lot of you like it, but if it breaks the app and there's no fix, then it has to go.

UPDATE 2009-02-25: As it turns out, we had most of the above fix in already (except that we were using a delay, not a timer—not a problem, since it's on its own thread). So the fix was simply to move most of the code to the main thread. I've done that and it works. The relevant patches are pending review; if I hear nothing bad about them from the other developers by tomorrow, I'll make them permanent commits and push them to both repositories, where they will be part of 1.1.5.

Other points

This is not a bug in WebKit. Strictly speaking, it's not a bug in GrowlMail, either, because it's not like we're disobeying the documentation for Mail's API (there is none!).

Apple changed WebKit to throw this exception, and GrowlMail doesn't catch it. As far as I'm concerned, it's GrowlMail's fault and we're the ones who need to fix it.

Timeline? No idea. I do intend to have this fix in for 1.1.5, since Safari 4 will probably be either out or coming Real Soon Now by then. (This among other important fixes, such as the off-by-two error in the Growl prefpane.)

UPDATE 2009-06-22: The fix is in GrowlMail 1.1.5, which we released on 2009-06-16.

Consider this code:

enum { foo, bar, baz } my_var;
switch (my_var) {
    case foo:
        int foo_alpha; //Line 7
        int foo_beta;
        break;

    case bar:
        my_var = baz;
    case baz:
        printf("Bar or baz encountered\n");
        break;
}

Try to compile it as C99.

Here's what GCC says:

test.c: In function ‘main’:
test.c:7: error: syntax error before ‘int’

What!

First off, why not? What is invalid about this declaration statement?

Second, why is foo_alpha invalid and not foo_beta?

There are several definitions in the C99 specification that come together to cause this problem.

The first is that there is no such thing as a declaration statement, because declarations are not statements. See §6.7 and §6.8; note that neither definition includes the other. In the language that is C99, declarations and statements are separate concepts.

The second is the definition of a compound statement. The definition of a switch statement (which is part of §6.8.4) is:

switch ( expression ) statement

If you go back up to §6.8, you'll see that another possible kind of statement is a compound statement, for which §6.8.2 gives this definition:
compound-statement:
  • {   block-item-listopt   }
block-item-list:
  • block-item
  • block-item-list   block-item
block-item:
  • declaration
  • statement

So a declaration is not a statement, a compound statement can contain declarations and/or statements, and a switch statement is a prefix upon (usually) a compound statement.

Now, the kicker. Read the relevant definition of a labeled statement from §6.8.1:

case   constant-expression   :   statement

Statement. Not block-item. Not declaration. Statements only.

So this is what the compiler sees in valid code (with a declaration not following a case label):

  • selection statement (switch)
    • compound statement
      • labeled statement
        • statement
      • statement
      • statement
      • declaration
      • statement
      • jump statement (break)
      • labeled statement
      • statement
      • jump statement (break)
      • labeled statement
      • statement
      • jump statement (break)

Now, consider how the compiler sees my code above:

  • selection statement (switch)
    • compound statement
      • labeled statement
        • declaration — wait, this isn't a statement! ERROR
      • declaration — also a kind of block-item, so it's perfectly valid here
      • jump statement (break)
      • labeled statement
      • statement
      • labeled statement
      • statement
      • jump statement (break)

I hope this makes clear that this isn't a compiler bug; the C99 language really does work this way.

(One possible solution would be to make a declaration a kind of statement, but I don't know what other ramifications that might have. [UPDATE 11:36: Jeff Johnson tells us why not.])

Half-Life 2 Photography

 

2009-02-19 13:14:46 UTC

My challenge to you:

Take a screenshot in any of the Half-Life 2 games (including Portal), and make it look and feel like an artistic or journalistic photograph. (I don't mean filters; I mean framing and the scene itself.)

Here's my first contribution:

A scene of the observer looking out a grimy window on to a Combine autogun emplacement.

If you want to join in, take a photographic-looking screenshot, and submit it to the Flickr group.

Manpage Monday: CC_SHA(3cc)

 

2009-02-16 05:41:54 UTC

The Common Crypto library in Mac OS X 10.4 and later provides simple APIs for five SHA algorithms:

CC_SHA1() computes the SHA-1 message digest of the len bytes at data and places it in md (which must have space for CC_SHA1_DIGEST_LENGTH == 20 bytes of output). It returns the md pointer.

CC_SHA1_Init() initializes a CC_SHA1_CTX structure.

CC_SHA1_Update() can be called repeatedly with chunks of the message to be hashed (len bytes at data).

CC_SHA1_Final() places the message digest in md, which must have space for CC_SHA1_DIGEST_LENGTH == 20 bytes of output, and erases the CC_SHA1_CTX.

The successor versions of SHA-1, SHA-2, are also implemented for hash bit lengths of 224, 256, 384, and 512. The functions to call to invoke the larger hash-size versions of the algorithms include the hash size as part of the function names: …

RETURN VALUES

All routines return 1 except for the one-shot routines ( CC_SHA1(), etc.), which return the pointer passed in via the md parameter.

A note about ClickToFlash 1.2

 

2009-01-31 08:58:42 UTC

Christopher Bowns pointed out (thanks!) that Jonathan Rentzsch released ClickToFlash 1.2 yesterday.

This is the first version that incorporates any significant features from my fork. Specifically:

  • It now loads the Flash movie on mouse-up, not mouse-down.
  • It now draws the ClickToFlash view as concave when the mouse is down and within its bounds.
  • It now has a separator item in the contextual menu.

The first two of these came from my tree, a fact that I am very happy about. The last one he pulled from Troy Gaul's tree instead, but it makes no difference, as it's just an element in a xib document.

If you're using 1.1+boredzo, you don't need to upgrade. My version of 1.1 has all the features of his 1.2, plus the “Copy Address” contextual menu item and, of course, my own click-to-play symbol.