Archive for April, 2009

Scale

Wednesday, April 29th, 2009

264 doesn't sound much bigger than 232, does it?

264 is 232 × 232, but this relation is hard to put into perspective. Even saying that 264 is more than four billion times as big as 232 doesn't adequately convey how much larger it really is.

Let me do that for you now.

I wrote a program that counts from 0 to 232. It takes about 11 seconds to run:

% ./count-up
2009-04-29 09:52:55.387 count-up[57932:10b] Time to count up to 4294967295: 11.313282

Now, as I said, 264 is 232 × 232, so the time to count to 264 is likewise the time to count to 232 (11.313282 seconds) multiplied by 232. That works out to:

  • About 48,590,176,200 seconds
  • About 13,497,271 hours
  • About 562,386 days
  • About 1540 years

Simply by doubling the exponent, we increase the time it would take my late-2006 four-core Mac Pro to count up to the number from 11 seconds to a millenium and a half.

Symbolicator 1.0.1

Friday, April 24th, 2009

It's two-thirds faster, it works on more crash logs, and thanks to Augie Fackler, it's now available from the Python Package Index. This means that you can just do this:

sudo easy_install symbolicator

If [you run a custom build of Python and] you don't want to install setuptools, you can always get the Symbolicator from its webpage.

Oh, and there will be a 1.0.2 to fix some issues. At the moment, I'm going back to working on Growl things, so I won't start on those for awhile; if you want to beat me to them, fork the project on Bitbucket, commit your fixes, and send me a pull request. (Make sure you co-ordinate your efforts on the ticket! I don't want to have to choose from two independently-developed fixes.)

UPDATE 2009-04-25: Clarified above that installing setuptools is only necessary if you have installed Python yourself. Thanks to Augie for pointing this out in his comment.

How to make hg merge, hg resolve use FileMerge

Wednesday, April 22nd, 2009

Put this in your ~/.hgrc file:

[merge-tools]
filemerge.executable=opendiff
filemerge.args=$other $local -ancestor $base -merge $output
filemerge.gui=True

(Based on the original description of and Matt Mackall's comment on a Mercurial bug about merging.)

New tool: The Symbolicator

Sunday, April 19th, 2009

The most significant behind-the-scenes change in Growl 1.1.5 is that we now build using the DWARF-with-dSYM debug symbol format.

Now, we distribute a fully-stripped executable for every release—even betas—and we keep the dSYM bundles on hand separately. The idea here is that we can use the dSYM bundles to obtain symbolic information (function name, filename, and line number) for the bare addresses in users' crash logs.

Unfortunately, there are no good tools for symbolicating Mac OS X crash logs. If this were an iPhone app, we could just drag the crash logs into the Xcode Organizer window, but Xcode doesn't do this for Mac crash logs.

I tried all the other tools as well, and every one of them had one of two problems:

  1. Didn't work at all
  2. Needed the dSYM bundle and main bundle to be next to each other on disk

#2 is not a deal-breaker, but it is a hassle.

So I wrote my own symbolication tool.

The Symbolicator is a Python script that:

  1. Reads in a crash log.
  2. Finds the necessary dSYM bundles using Spotlight. (This means that you can put the dSYM bundles anywhere you want, and as long as Spotlight can find them, the Symbolicator will be able to use them.)
  3. Uses dwarfdump to extract the relevant symbol information.
  4. Replaces the address offsets with the symbol information (just like CrashReporter does when it has debug symbols to work with).
  5. Writes the symbolicated log to stdout.

Use it like this:

% symbolicator < unsymbolicated.crash > symbolicated.crash

Or use ThisService to make a service out of it.

If you want to see it in action right now, download Growl 1.1.5b1 and the corresponding dSYM bundles from the Growl beta page. Make Growl crash (killall -TRAP works well), then unpack the bundles and use the Symbolicator on the crash log. (If you unpack the bundles first, CrashReporter will symbolicate the log before you even get to it. Handy, unless you're trying to test the Symbolicator. ☺)

To make this work on your own app, follow the instructions in the aforementioned ADC article, then make sure you archive the dSYM bundles for every release, including betas. On Growl, I added code to our Release Makefile for this.

Note: If your app is closed-source, you should not put your dSYM bundles on your website, since the debug symbols are arguably trade secrets (information about your source code). Keep them locally, perhaps on a flash-memory drive. Disclaimer: IANAL.

Manpage Monday: PlistBuddy(8)

Monday, April 13th, 2009

PlistBuddy(8) is a command-line tool for editing property-list files, with a deeper reach than plutil defaults:

Entries consist of property key names delimited by colons. Array items are specified by a zero-based integer index. Examples:

  :CFBundleShortVersionString
  :CFBundleDocumentTypes:2:CFBundleTypeExtensions

Jonathan “Wolf” Rentzsch, in his del.icio.us bookmark for the manpage, says that PlistBuddy is “always at /usr/libexec/PlistBuddy on modern systems”.

Don’t pass objects as context pointers

Friday, April 3rd, 2009

Cocoa has several functions and methods that run a panel (usually as a sheet) asynchronously. These functions and methods take a target object, a selector, and a context pointer, and when they finish, they send a message to the target with that selector, passing that context pointer.

A common practice is to pass an object in the context pointer. You retain the object before you call the run-panel function or method, then release it in your callback. For example:

    NSBeginAlertSheet(/*title*/ NSLocalizedStringFromTableInBundle(@"Update Available", nil, bundle, @""),
                      ⋮
                      /*modalDelegate*/ self,
                      /*didEndSelector*/ NULL,
                      /*didDismissSelector*/ @selector(downloadSelector:returnCode:contextInfo:),
                      /*contextInfo*/ (void *)downloadURL,
                      …);

In this example, downloadURL is a local variable, and the object in it was retained above, or maybe not autoreleased in the first place. The downloadSelector:returnCode:contextInfo: method will cast the context pointer back to an object type, probably do something with it, then release it.

This has recently become a visible problem because of the Clang Static Analyzer. The analyzer doesn't know that NSBeginAlertSheet will retain the object, so it flags this call as a leak. This is the most-often-cited example of the checker's “false positives”: things that it flags as bugs but that aren't really bugs.

But is it really not a bug? Is it really appropriate to use an object as a context pointer?

Let's think about what we're doing here:

  1. The panel doesn't know the context pointer is really an object, so it doesn't retain it. To compensate for this, we're retaining an object on behalf of another object (the panel). This alone should make you feel dirty.

  2. The panel doesn't know the context pointer is really an object, so it won't release the object, either. This is particularly a problem if something releases the panel (quite possible in the case of, say, an NSAlert instance). Then it really is a leak.

  3. If we get released (and, hopefully, take the panel with us or at least cancel it), then we leak the object, because we are only expecting to have to release it (or even to have access to it!) in the callback.

Passing objects through context pointers is an anti-pattern. I hope the Clang Static Analyzer developers never remove this warning, because this is not a false bug—it is a real bug that we should all fix in our programs.

Solutions

That object needs to go in an instance variable somewhere.

The lightweight solution is to create an instance variable on the object that is running the sheet. You store the object in the instance variable when you run the sheet (and leave the context pointer NULL), then retrieve the object from the instance variable in your callback. You'll still need to retain it when putting it in, and release it both in your callback and in dealloc—those aspects don't change.

I can hear you saying that this is a waste of an instance variable. What are they, rationed? There's nothing to waste; instance variables are cheap. And leaking objects is far more of a waste of memory than adding an instance variable.

But this isn't the only solution.

Another way is to create your own subclass of the panel class (NSOpenPanel, NSSavePanel, or NSAlert), and give the subclass a property you can use instead of the context pointer. You could make it abstract (making the property similar to NSMenuItem's representedObject, nothing more), or make it specific to your application. The former approach is simpler, whereas the latter gives you a base on which to build a more-customized alert if you later decide you want one. It's up to you.

The advantage of this solution is that, since you've formalized the instance variable as part of a class's API (and since the instance variable is no longer a bit player but the central part of a class), it'll be harder for you to forget to manage your ownership of it correctly. Memory problems solved.

One way or another, though, you need to put that object into an instance variable somewhere to avoid leaking memory, feeling dirty, and having the Clang Static Analyzer chew you out.

(And yes, all this applies with garbage collection as well. See the section “GC Gotchas” in episode 36 of Late Night Cocoa. However, I disagree with Andre Pang's recommendation in that episode of CFRetain and CFRelease; the right solution is not to dip into Core Foundation, but to avoid the context pointer altogether and use an instance variable instead.)

hg precommit hooks and the Clang Static Analyzer

Friday, April 3rd, 2009

Fraser Speirs has a post about configuring your Git repository to vet commits with the Clang Static Analyzer.

The idea of pre-commit hooks is that you get to run a script before the commit happens. Depending on the result code of the script, the commit will either proceed or be aborted.

I wrote a wrapper around the scan-build tool, so that I could run the analyzer by hand with my preferred options at any time: …

The --status-bugs flag is the trick here: it makes scan-build return a non-zero status code if it detects bugs. That’s what we want with Git pre-commit hooks: a non-zero status indicates a possible bug, and that causes the Git commit to be aborted.

Mercurial, of course, has the same feature. The hg book has instructions; I'll show you how I set up my repository to do this.

First, I created a shell script named RunClang.zsh:

#!/bin/zsh -f
~/bin/checker-latest/scan-build \
    -checker-cfref -warn-objc-methodsigs -warn-objc-missing-dealloc -warn-objc-unused-ivars \
    --status-bugs -o checker.out \
    xcodebuild -configuration $1

Next, I added my precommit hook to the repository's .hg/hgrc file:

[hooks]
precommit.RunClang = ~/bin/RunClang.zsh Development

Here's what an example session looks like:

% echo 'Testing precommit testing hook' >> test.txt
% hg ci -m 'Testing precommit testing hook'
[churn churn churn]
** BUILD SUCCEEDED **
scan-build: 17 bugs found.
scan-build: Run 'scan-view [snip]/growl-boredzo-precommit-test/checker.out/2009-04-03-2' to examine bug reports.
abort: precommit.RunClang hook exited with status 1

(255)% hg log --limit=1
changeset:   4188:b208862a586d
tag:         tip
user:        Peter Hosey
date:        Fri Mar 13 05:40:09 2009 -0700
summary:     Fix encoding of the Norwegian Growl-WithInstaller strings file.

17 bugs—mostly leaks. Glad I didn't commit this test file!

So now you know how to have Mercurial block you from committing if the clang checker can find bugs. This should also work for your unit tests. And if you have 100% test coverage (lucky!), you can combine them: have scan-build build your test-bundle target. Then, the hook will prevent the commit if the checker can find bugs or any tests fail.

I don't think I'll actually use this set-up, though.

First, in order to find bugs, you need to build your entire main product. Any significantly large program is going to take a long time to build and analyze—Growl, for example, takes about one-and-a-quarter minutes for a clean build. Even committing to a Subversion repository over dial-up was quicker.

More significantly, precommit hooks like this interfere with patch queues. The mq extension implements patches as mutable commits, so any qnew or qrefresh will run the hook. It would be useful on qfinish, but it's just annoying on qnew and especially qrefresh, as the all-too-frequent builds thwart rapid iteration.

So, if you use patch queues, this won't work for you. But, if you don't, then this should work as well in Mercurial as it does in Git.

Adding Growl support to Mercurial

Wednesday, April 1st, 2009

Add this to your ~/.hgrc file:

[hooks]
changegroup.growl = ((echo "$HG_URL" | grep -vF 'file:' > /dev/null) && growlnotify -n Mercurial 'Pull successful' -m "Pulled at least $(hg log --template='\n' -r$HG_NODE:tip | wc -l | sed -e 's/ //g') changesets from $HG_URL") || true
outgoing.growl = (test "x$HG_URL" '!=' 'x' && growlnotify -n Mercurial 'Push successful' -m "Pushed to $HG_URL") || true

It's not perfect: Both notifications share the same notification name. As long as that isn't a problem, this works fine.