Mentoring
Tuesday, March 8th, 2011I originally wrote this for the Adium development list, in a discussion of the upcoming Google Summer of Code, for the benefit of those of our developers who have never mentored students in GSoC before and may be considering doing so. Read this in that context; for example, “our” means “the Adium project’s”.
-
Work with students from the very beginning on the quality of their [application] submissions. 90% of the submissions will be crap. Look past this. See what lies within. If it can be improved, try to get them to improve it. Those who do will be good students.
-
Spur your student to work on code. Make sure they’re committing at a regular and satisfactory rate.
-
Ensure your student actually writes code and doesn’t just crib together tutorials and/or Stack Overflow answers and/or ask you to provide code* for various tasks they need to accomplish. Wildly varying coding styles, inconsistent variable and method naming, and poor indentation are warning signs.
-
Communicate constantly. This includes the aforementioned spurring your student to work as well as being available to answer any questions they have about where things are in our code base, what they need to do to achieve certain goals or specific sub-tasks, etc. These questions are virtuous; you should encourage them, just short of demanding them. The student is not that only in name; they are here both to write code and to learn.
-
Find a medium that works for both of you. For me, email (or Twitter, nowadays) would be best. Maybe you prefer IM, or even voice chat/phone (if it isn’t too expensive). Don’t try to force your habits on the student; if you love the phone but they hate it (or vice versa), a happy medium will work better.
-
Be sure they understand and use Mercurial and good VCS practices generally. Frequent, discrete commits; neither waiting “until it’s done” to commit (it should compile) nor committing half-done work periodically (e.g., daily) nor committing amalgams of unrelated work (they should commit specific sections that comprise a single change).
-
Nowadays, I recommend that you have them fork our Bitbucket repo and push to their fork.
-
Review their code constantly. Subscribe to the commits list (or their fork’s commits feed) and review everything they write.
-
Read their commit messages, not just their code. Lists are a warning sign (unrelated changes lumped together). Inadequately describing the change is also a problem. Work the student out of these habits as soon as possible.
-
Don’t wait until mid-term exams to sit your student down for a serious talk about their work or lack thereof. If they’re committing garbage, set them straight as soon as possible—do not wait. If they don’t commit, get them writing and committing as soon as possible.
-
Always be ready to fail your student. Be compassionate, understanding of life’s realities; they’re not a slave. But they are here to work, and if they don’t do the job or if they do a bad job, be ready to fail them.
-
Make sure they know where they stand. If there’s 1–2 weeks before exams and they’re still in danger of failing, make sure they know what’ll happen if they don’t shape up.
I can’t claim to have been perfect in all of these points in my own mentoring (many of these I learned by not doing them), but it’s what I’ve found works.
There might also be something on the GSoC sites about this. Some viewpoints vary, particularly along the leniency-to-hardassness spectrum.
If you are not willing to do all of this, or don’t think you’ll have the time, you should not mentor a student.
* Six words that should worry every mentor or other help-offerer: “Can you show me a sample?” ↶
svn merging is full of fail
Thursday, August 7th, 2008[SCENE: adium/branches/summer_of_code_2008/unit_testing. I’m merging changes from trunk to this branch.]
svn --version svn, version 1.5.0 (r31699) compiled Jul 5 2008, 04:29:25 Copyright (C) 2000-2008 CollabNet. Subversion is open source software, see http://subversion.tigris.org/ This product includes software developed by CollabNet (http://www.Collab.Net/). The following repository access (RA) modules are available: * ra_neon : Module for accessing a repository via WebDAV protocol using Neon. - handles 'http' scheme - handles 'https' scheme * ra_svn : Module for accessing a repository using the svn network protocol. - with Cyrus SASL authentication - handles 'svn' scheme * ra_local : Module for accessing a repository on local disk. - handles 'file' scheme * ra_serf : Module for accessing a repository via WebDAV protocol using serf. - handles 'http' scheme - handles 'https' scheme ___ svn up At revision 24694. ___ svn st ? Docs ? DoxyCleaned ~ Resources/Message Styles/renkooNaked.AdiumMessageStyle/Contents/Resources/Incoming/buddy_icon.png ~ Resources/Message Styles/renkooNaked.AdiumMessageStyle/Contents/Resources/outgoing_icon.png ~ Resources/Message Styles/renkooNaked.AdiumMessageStyle/Contents/Resources/Outgoing/buddy_icon.png ~ Resources/Message Styles/renkooNaked.AdiumMessageStyle/Contents/Resources/incoming_icon.png (~ = “versioned item obstructed by some item of a different kind”) rm -Rf Resources/Message\ Styles/renkooNaked.AdiumMessageStyle ___ svn up Resources/Message\ Styles/renkooNaked.AdiumMessageStyle [snip] ___ cd ../../.. ___ svn merge -r23936:24694 trunk branches/summer_of_code_2008/unit_testing --- Merging r23937 through r24694 into 'branches/summer_of_code_2008/unit_testing': [snip] Conflict discovered in 'branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist'. Select: (p) postpone, (df) diff-full, (e) edit, (h) help for more options: e [I correct the plist to match what trunk has.] Select: (p) postpone, (df) diff-full, (e) edit, (r) resolved, (h) help for more options: r ___ plutil -lint branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist: OK ___ plutil -lint branches/summer_of_code_2008/unit_testing/Adium.xcodeproj branches/summer_of_code_2008/unit_testing/Adium.xcodeproj: file does not exist or is not readable or is not a regular file ___ plutil -lint branches/summer_of_code_2008/unit_testing/Adium.xcodeproj/project.pbxproj branches/summer_of_code_2008/unit_testing/Adium.xcodeproj/project.pbxproj: OK ___ plutil -lint branches/summer_of_code_2008/unit_testing/Frameworks/AIUtilities\ Framework/AIUtilities.framework.xcodeproj/project.pbxproj branches/summer_of_code_2008/unit_testing/Frameworks/AIUtilities Framework/AIUtilities.framework.xcodeproj/project.pbxproj: OK ___ cd branches/summer_of_code_2008/unit_testing %~/Projects/@otherpeoplesprojects/adium(0) ___ xcodebuild -configuration Development [snip] ** BUILD SUCCEEDED ** ___ svn ci Sending unit_testing [snip] Transmitting file data .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................svn: Commit failed (details follow): svn: Checksum mismatch for '/Volumes/Home-etc/Users/prh/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset/.svn/text-base/Emoticons.plist.svn-base'; expected: '109d361ce47a49a66e53e98412251398', actual: 'a21c8336d1f839b9aaa8a2795a7e25f5' svn: Your commit message was left in a temporary file: svn: '/Volumes/Home-etc/Users/prh/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008/svn-commit.2.tmp' ___ cd Resources/Emoticons ___ rm -Rf MSN.AdiumEmoticonset ___ svn up MSN.AdiumEmoticonset A MSN.AdiumEmoticonset [snip] Updated to revision 24694. ___ cd ../.. ___ cd ../../.. ___ svn merge -r23936:24694 trunk branches/summer_of_code_2008/unit_testing svn: Working copy path 'Frameworks/Adium Framework/Source/AISharedAdium.h' does not exist in repository ___ svn merge -r23936:24694 trunk/Resources/Emoticons branches/summer_of_code_2008/unit_testing/Resources/Emoticons ___ svn merge -r23936:24694 trunk/Resources/Emoticons/MSN.AdiumEmoticonset branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset ___ svn merge -r23936:24694 trunk/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist ___ cd branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset ___ svn pl Emoticons.plist ___ svn st ___ popd ___ diff -u trunk/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist --- trunk/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist 2008-06-26 20:50:51.000000000 -0700 +++ branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist 2008-08-07 11:20:27.000000000 -0700 @@ -367,7 +367,7 @@Name Cool -I don't know.gif +I Don't Know.gif Equivalents [Um, OK. What happened to all the differences it had from trunk that I had merged over? Suddenly, almost all the differences are gone, and the files are nearly the same!] ___ diff -u trunk/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist branches/summer_of_code_2008/unit_testing/Resources/Emoticons/MSN.AdiumEmoticonset/Emoticons.plist ___ cd branches/summer_of_code_2008 %~/Projects/@otherpeoplesprojects/adium(0) ___ svn ci unit_testing Sending unit_testing Transmitting file data ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................svn: Commit succeeded, but other errors follow: svn: Error bumping revisions post-commit (details follow): svn: Directory '/Volumes/Home-etc/Users/prh/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008/unit_testing/Frameworks/Sparkle.framework/Versions/A/Resources/ja.lproj/SUUpdateAlert.nib' is missing svn: Directory '/Volumes/Home-etc/Users/prh/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008/unit_testing/Frameworks/Sparkle.framework/Versions/A/Resources/ja.lproj/SUUpdateAlert.nib' is missing svn: Your commit message was left in a temporary file: svn: '/Volumes/Home-etc/Users/prh/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008/svn-commit.3.tmp' ___ svn st %~/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008(1) ? svn-commit.2.tmp ? svn-commit.tmp ? svn-commit.3.tmp ML unit_testing ? unit_testing/Docs ? unit_testing/DoxyCleaned L unit_testing/Release L unit_testing/Release/openUp L unit_testing/Release/Artwork M unit_testing/Release/Makefile L unit_testing/Frameworks L unit_testing/Frameworks/libgmodule.framework L unit_testing/Frameworks/libgmodule.framework/Versions [snip] (L in third column = Locked) ___ svn cleanup %~/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008(0) svn: In directory 'unit_testing/Frameworks/JSON Framework/Tests/json.org' svn: Error processing command 'committed' in 'unit_testing/Frameworks/JSON Framework/Tests/json.org' svn: Working copy 'unit_testing/Frameworks/JSON Framework/Tests' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) ___ svn cleanup %~/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008(1) svn: In directory 'unit_testing/Frameworks/JSON Framework/Tests/json.org' svn: Error processing command 'committed' in 'unit_testing/Frameworks/JSON Framework/Tests/json.org' svn: Working copy 'unit_testing/Frameworks/JSON Framework/Tests' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) [So I get to manually clear the locks. Yay!] ___ tar cjf unit_testing.tbz unit_testing ___ cd unit_testing %~/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008(0) ___ svn st | grep -Fw L | head L Frameworks L Frameworks/JSON Framework L Frameworks/JSON Framework/Tests A L+ Frameworks/JSON Framework/Tests/json.org A L+ Frameworks/JSON Framework/Tests/types A L+ Frameworks/JSON Framework/Tests/format A L+ Frameworks/JSON Framework/Tests/rfc4627 L Frameworks/JSON Framework/Tests/Examples L Frameworks/JSON Framework/Tests/Examples/JSONChecker A L+ Frameworks/JSON Framework/Tests/jsonchecker svn: Write error: Broken pipe ___ ls Frameworks/.svn dir-prop-base format prop-base/ text-base/ entries lock props/ tmp/ ___ ls Frameworks/.svn/lock Frameworks/.svn/lock ___ rm Frameworks/.svn/lock ___ svn st -N Frameworks L Frameworks/JSON Framework L Frameworks/libintl.framework L Frameworks/libpurple.framework L Frameworks/libmeanwhile.framework L Frameworks/LMX.framework R L+ Frameworks/Sparkle.framework L Frameworks/AutoHyperlinks Framework L Frameworks/libgobject.framework L Frameworks/OTR.framework L Frameworks/libglib.framework L Frameworks/ShortcutRecorder L Frameworks/libgthread.framework L Frameworks/AIUtilities Framework L Frameworks/OCMock.framework L Frameworks/PSMTabBarControl.framework L Frameworks/RBSplitView L Frameworks/Growl-WithInstaller.framework L Frameworks/Adium Framework ___ svn st -N Frameworks/JSON\ Framework L Frameworks/JSON Framework A + Frameworks/JSON Framework/dmg.sh L Frameworks/JSON Framework/Tests L Frameworks/JSON Framework/Site L Frameworks/JSON Framework/JSON.xcodeproj L Frameworks/JSON Framework/Docs A + Frameworks/JSON Framework/bench.m M Frameworks/JSON Framework/JSON-Info.plist M Frameworks/JSON Framework/CREDITS ___ rm Frameworks/JSON\ Framework/.svn/lock ___ svn st -N Frameworks/JSON\ Framework A + Frameworks/JSON Framework/dmg.sh L Frameworks/JSON Framework/Tests L Frameworks/JSON Framework/Site L Frameworks/JSON Framework/JSON.xcodeproj L Frameworks/JSON Framework/Docs A + Frameworks/JSON Framework/bench.m M Frameworks/JSON Framework/JSON-Info.plist M Frameworks/JSON Framework/CREDITS ___ find . -name lock -print0 | xargs -0 rm ___ svn ci svn: Working copy '/Volumes/Home-etc/Users/prh/Projects/@otherpeoplesprojects/adium/branches/summer_of_code_2008' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) ___ rm ../.svn/lock ___ svn ci Sending unit_testing [snip] svn: Commit failed (details follow): svn: Directory '/branches/summer_of_code_2008/unit_testing' is out of date ___ svn up .. svn: Failed to add file '../unit_testing/Frameworks/JSON Framework/dmg.sh': a file of the same name is already scheduled for addition with history ___ rm Frameworks/JSON\ Framework/dmg.sh ___ svn up D Frameworks/libgmodule.framework/Versions/2.0.0/Resources/English.lproj G Frameworks/JSON Framework/Tests/Types.m svn: Failed to add directory 'Frameworks/JSON Framework/Tests/json.org': a versioned directory of the same name already exists ___ svn st Frameworks/JSON\ Framework/Tests/json.org A L+ Frameworks/JSON Framework/Tests/json.org A + Frameworks/JSON Framework/Tests/json.org/1.json A + Frameworks/JSON Framework/Tests/json.org/2.json A + Frameworks/JSON Framework/Tests/json.org/1.plist ⋮ ___ find . -name lock -print0 | xargs -0 rm ___ svn st Frameworks/JSON\ Framework/Tests/json.org A + Frameworks/JSON Framework/Tests/json.org ⋮ ___ cd .. ___ mv unit_testing unit_testing-fail ___ svn up unit_testing A unit_testing
With this fresh working copy, I’m able to build, so I believe I’m finally done.
A possible solution to the stolen-focus problem
Wednesday, December 5th, 2007Jeff Atwood posted an article today titled Please Don’t Steal My Focus, about dialog boxes coming up while you’re typing.
The problem, in a nutshell, is that these dialog boxes steal keyboard focus. Sometimes it’s for a text field; sometimes it’s just for Full Keyboard Access (i.e., a button gets the focus—usually, these are alert boxes). Regardless of what control gets keyboard focus, it gets it by stealing it from your app, which is a problem if the user is typing when that happens. Most likely, either some text will go to the wrong place, or the dialog box will beep for every keystroke; either result will annoy the user.
We’ve had the same problem for a long time in Adium: we’ll throw up a dialog box, but the user is typing a message, and notices that half his message isn’t in the inputline. We believe we’ve fixed most of these, but we still get complaints about this problem from time to time.
Currently, we solve the problem by simply ordering the dialog front without making it key. This works for most people, but it doesn’t help people who really do need Full Keyboard Access, since you have to click on the dialog box to dismiss it (or make it key) if it isn’t key. We (by which I mean Mac programmers, not just Adium) need a better solution.
I propose an addition to NSAlert. Here’s how it would work:
- From the creation of the NSAlert instance, it watches for key events.
- When any key goes down, the alert resets the timer (if one is present) and sets a flag.
- When all keys are up, the alert creates and starts the timer (with an interval of, say, a couple of seconds).
- If you tell the alert to show, it checks the flag. If the flag is clear, it simply performs
makeKeyAndOrderFront:
like usual. If the flag is set, it only orders front, and it sets a second flag. - When the timer fires, the alert clears the first flag, and if the second flag is set, makes itself key.
From the user’s perspective, this results in the alert waiting a couple of seconds for them to pause in their typing before it takes keyboard focus.
Does anybody know of an existing free implementation of this, or plan to write it themselves?
Thanks
Friday, November 30th, 2007- To Brent Simmons and Mike Lee.
- To Matt Drance, who gave me a ride back to my hotel tonight.
- To Colin Barrett, who gave me a ride back to my hotel (more or less) on Tuesday. (We did not go directly to the hotel.)
- To Bill Bumgarner, who brought 33 pounds of delicious pork. I think our row of tables needed two more to seat all the people who came for it.
- To Blake C., who is still the biggest fan of this blag.
- To everybody, who knew exactly what I was talking about every time I said “blag”. I loved that.
- To Wil Shipley and Brent for picking up a couple of tabs.
- To Brent, for supplying me with Yellow Cab’s phone number, so that I could call us a cab back to the hotel. Also for paying the fare.
- To John Geleynse for the invitation, and to Evan Schoenberg for forwarding it on.
- To everybody who helped me with stuff at the event (including, but not limited to, Deric Horn, Michael Jurewitz, and Peter Ammon).
Let me know if you did something for me this week and I forgot to thank you, or if you want me to remove you from the list for some reason.
I’ve been interviewed!
Monday, October 8th, 2007I don’t ordinarily post to my main blog with a link, but this isn’t my average link. ☺
Jonathan Allen (also known as grauenwolf on reddit) interviewed me about how I do code review on Adium, after I mentioned it in a comment thread on programming.reddit.
In the interview, I talk about why you should review your fellow developers’ code and how it works on Adium. I say it’s worth reading for anybody who works on a software project with other people.
My first unit tests
Friday, August 24th, 2007Last night, I added unit tests to Adium. Despite the fact that I have espoused adding tests for some time (usually when stuff broke that would have been caught by an automated test), I actually had never created tests before.
I love it.
Here’s why I added tests, and my quick overview of how to do it.
Why
I started out looking at fixing #7713. The problem was that our display of intervals was including units of which there were zero: the example from the ticket is “1 Week 4 Days 0 Hours 43 Minutes”. (We don’t really capitalize those letters, in case you’re wondering.)
Upon investigating, I found that the method that creates these strings was complex and thinly commented, and it didn’t work in the obvious way (building an array and then joining its elements). The upshot of that was that I could guess that that the method was probably deeply broken, but in such ways that made it not obvious exactly how it was broken. This left me uncertain of how to proceed, since I had the sneaking feeling that any change had a chance of breaking something else, even if it fixed #7713.
What I needed was a complete set of diagnostics that would attempt to extract every combination of time units and make sure that all of the results were in line with my expectations.
In other words—a test suite!
The major mental block that had kept me from getting into unit testing had always been how tautological it seemed: having just written some code that does X, you then write code that makes sure X got done. This seemed redundant, mostly because I had been envisioning assigning to a variable, then checking the value in the variable to make sure it really got assigned. (Obviously, that’s wrong. I never seriously thought that people were really doing that, but couldn’t think of what else you would test.)
Now, I had a case that clearly and finally proved that real unit testing isn’t anywhere near tautological: I had code that’s broken, that I could break in some other way by attempting to fix it. Thorough testing would help me find all the ways in which it’s broken, so that I could fix it without breaking something else.
This is exactly what unit testing is for.
How
Here’s a brief summary of what you need to do to set up your project the unit tests (assuming that you’re using Xcode, version 2.1 or later):
-
Create one or more subclasses of SenTestCase.
Here’s what one of our tests looks like:
- (void)testDateFormatterStringRepWithInterval_minutes { //Note that we want a date in the past, because we want to describe the time interval since that date. NSDate *date = [[NSCalendarDate calendarDate] dateByAddingYears:-0 months:-0 days:-0 hours:-0 minutes:-10 seconds:-0]; STAssertEqualObjects([NSDateFormatter stringForTimeIntervalSinceDate:date], @"10 minutes", @"Unexpected string for time interval"); }
Every individual test should be in its own method in your class, and that method’s selector must begin with “test”. Go wild—you don’t need to list them anywhere, because the test runner detects them all automatically (hence the selector-begins-with-“test” requirement). So don’t worry about having lots and lots of methods, as long as they’re appropriately categorized in different SenTestCase subclasses (for the sake of small, easily-navigable files).
And yes, you do need to subclass SenTestCase, as it provides methods that the STAssert… macros need.
-
Add a Cocoa/Unit Test Bundle target to your Xcode project.
-
Add your SenTestCase subclasses to the bundle target.
You probably should not put the test case class in your main application or framework targets. I know I can’t think of a reason why you would.
-
Configure the Info.plist of the bundle in the usual way (Get Info on the target, then set the bundle ID and signature on the Properties tab).
Henceforth, whenever you want to run your tests, build the bundle target. You don’t need to run it, because the last phase of the target is a shell script that runs the tests. This is so that you can make the test bundle target a dependency of your main application/framework target, in order to have it run whenever you try to build the app/framework.
(Also, this is the 500th post in my blog’s WP database. Woo-hoo!)
Fixing an Adium bug: A timeline
Tuesday, August 14th, 2007I’m fixing a hole, where the rain gets in/and stops my mind from wondering/where it will go-ooo…
All times are PDT.
-
2007-08-12 14:04:38: Evan releases 1.1 in the appcast. 1.1 is our first version that requires Tiger.
-
2007-08-12 14:08:09: Evan puts back 1.0.5 alongside 1.1, with a sparkle:minimumSystemVersion attribute on the enclosure element noting that 1.0.5 requires 10.3.0 or later.
-
2007-08-12 14:25:12: Evan changes 10.3.0 to 10.3.9; David had pointed out that 1.0.5 won’t run on OS versions before 10.3.9.
-
2007-08-12 17:32:02: First user report on the feedback address from a 10.3.9 user that she had updated to 1.1 and it didn’t run on her system.
-
2007-08-13, about 06:00: I wake up, and find six such user reports sent to the feedback list, including the first one. I answer them all saying that 1.1 requires Tiger and you can get 1.0.5 from the front page. By the sixth one, I’m suspicious that we may have a Sparkle bug.
-
06:31:48: I join the #adium IRC channel. I have a feeling that I’m going to need to field user reports there, too. I join the Adium development channel at the same time (my IRC client is set to join both automatically).
-
From the #adium IRC channel:
06:32:46: Mac-arena notes that the Adium 1.1 story is #1 in Apple, #4 in Technology, and #8 in News.
-
From the #adium IRC channel:
06:51:55: <proton> damn sparkle updating 10.3 users 06:53:59: <Mac-arena> Evan said he fixed it, which makes it weird.
-
07:57:58: I fix the appcast for Panther users. Sparkle actually looks for the minimum OS version in an element named sparkle:minimumSystemVersion, not in an attribute of the enclosure element.
-
08:13:58: I’m sending emails to Panther users who got the update notification before I fixed the appcast, then emailed our feedback address complaining that it didn’t work. Here’s an excerpt from eight such emails:
Adium 1.0.5 is available from the front page of http://adiumx.com/, at the bottom of the page. Unless some bug arises that we need to address quickly with a 1.0.6, this is the last version that will run on 10.3.9.
-
From the #adium IRC channel:
08:34:56: :Server: eventualbuddha (n=eventual@[deleted].sbcglobal.net) joined #adium 08:35:16: <eventualbuddha> congrats on 1.1 guys! in trying to update Adium it crashes though ⋮ 08:41:57: <Mac-arena> eventualbuddha: What version of Adium were you running at the time? 08:42:07: <eventualbuddha> 1.0.5 ⋮ 08:55:39: <eventualbuddha> Mac-arena, The_Tick: i should mention that the svn build i had was doing the same thing, but i just updated to 1.1 and it doesn't have that problem
So, this user has the problem with 1.0.5 and not 1.1.
-
From the #adium IRC channel:
08:53:58: :Server: gammah (n=gammah@[censored].swbell.net) joined #adium 08:54:28: <gammah> hey I was getting crashes in the previous version of adium, on what appeared to be the 'check for updates' function 08:54:46: <gammah> so I manually got adium, and it seems to suffer the same fate - crashing right at startup
This user has the problem with both 1.0.5 and 1.1.
-
09:22:11: Second ticket filed. The other tickets that will be filed about this are #7544, #7546, #7547, #7549, #7550, #7552, #7555, #7557, #7559, #7564, #7565, #7566, #7568. (Ever wonder why we don’t do more actual programming work on Adium? That’s why. However, to be fair, #7543 was filed on request from me, because we initially thought it to be a second bug.)
-
From the #adium IRC channel:
09:23:45: <Mac-arena> dang503: Interesting. 09:23:59: <Mac-arena> dang503: There are a lot of people getting this same crash on update in 1.0.5. 09:24:05: <Mac-arena> dang503: And you have it in 1.1.
-
From the #adium IRC channel:
09:47:17: <nphase> http://pastebin.com/m28849e83 09:48:51: <Mac-arena> Hey, let's see if we can get this crash posted to every pastebin on the internet. :D
-
By 10:00 it becomes clear on #adium that this is mostly happening to 1.1 people.
-
10:19:02: Received first email to the feedback address reporting the crash on update.
-
10:42: Posted to the blog acknowledging the crash and inviting users with debugger-fu to help us track it down.
-
From the Adium development IRC channel:
10:46:12: <The_Tick> zac: go into com.adiumX.adiumX.plist 10:46:21: <The_Tick> and change the last update time 10:46:23: <The_Tick> to today 10:46:29: <The_Tick> you'll be able to launch then
The other way to do this would have been defaults write com.adiumx.adiumx SULastCheckTime '2007-08-13 10:47:00 -0700'. The effect of this would have been to forestall the crash; it would not have actually fixed it. Zac says he did not perform the edit.
-
11:50:39: First volunteer joins IRC to help with debugging.
-
11:57:35: First debug build goes live.
-
12:02:05: I create a page on our wiki to keep the debug build link and provide instructions on how to step through and get a stack trace.
-
As evidence of how inaccurate the page was initially, this excerpt from #adium:
12:12:45: <prb> I have the build and XCode, but it's not obvious to me how to attach the debugger. What is the necessary dance to get the attach enabled? 12:12:59: <Mac-arena> prb: Build & Debug, cmd-y 12:13:45: <prb> Mac-arena: Is there a project lurking around somewhere, or do I need to download a source snapshot? 12:14:06: <Mac-arena> prb: Erm. Right. :D Probably easiest just to use gdb
-
From the #adium IRC channel:
12:16:34: <prb> I'm in -- anyone got a gdb cribsheet (catch foo, etc.) 12:16:40: <Mac-arena> prb: Writing the instructions now
-
Our webmaster, Zac, who is able to reproduce the crash (although at the time, it seems like a slightly different crash and is tentatively considered unrelated), posts his Console output, containing some NSLogs that I’d accidentally left in while diagnosing the earlier problem with the appcast (wherein the
minimumSystemVersion
was in the wrong place in the XML). I don’t realize it at the time, but the NSLogs actually give a hint to the problem, as they show the twominimumSystemVersion
objects each used once, and then the crash. -
From the #adium IRC channel:
12:25:32: <Mac-arena> Note for future reference 12:25:39: <Mac-arena> When you use cd Release && make 12:25:42: <Mac-arena> It uses Deployment [build configuration]. 12:25:48: <Mac-arena> OK, I get to do *THAT* over.
-
13:24:46: I post the link to the new debug build (this time actually compiled with Development build config)…
-
13:25:51: … and update the wiki page with it.
-
From the #adium IRC channel:
13:31:50: <efge> Mac-arena: Thread 0 Crashed: 13:31:50: <efge> 0 libobjc.A.dylib 0x90a594c0 objc_msgSend + 16 13:31:51: <efge> 1 org.ironcoder.SparklePlus 0x003eb552 -[SUUpdater newVersionAvailable] + 49 (SUUpdater.m:479) 13:31:51: <efge> 2 org.ironcoder.SparklePlus 0x003eb96b -[SUUpdater appcastDidFinishLoading:] + 786 (SUUpdater.m:507)
Line numbers! Woo-hoo!
So here’s line 478 (gdb being generally off by one with its line numbers):
BOOL available = SUStandardVersionComparison([updateItem minimumSystemVersion], SUCurrentSystemVersionString()) == NSOrderedDescending;
There’s only one message there, so that has to be what’s crashing.
Current suspicion: The
SUAppcastItem
is a zombie. -
About 13:36: I finally get the idea to try to reproduce the bug on my laptop.
-
From the #adium IRC channel:
13:38:25: <Mac-arena> Hey, everybody! 13:38:27: <Mac-arena> Good news 13:38:31: <Mac-arena> I can reproduce this on my laptop! 13:39:17: <durin42> yay Mac-arena can has crash
-
From the #adium IRC channel:
13:39:12: <Mac-arena> The appcast calls
-appcastDidFailToLoad:,
and we send itautorelease
, and that breaks. 13:40:52: <Mac-arena> Wait, that's wrong. The backtrace says-appcastDidFinishLoading:
and-newVersionAvailable
[accessor], so it's not in-DidFailToLoad:
. 13:40:59: <Mac-arena> I hate it when gdb lies to me. -
From the #adium IRC channel:
13:55:00: <Mac-arena> (gdb) po minimumSystemVersion 13:55:00: <Mac-arena> 1, 0 13:55:23: <Mac-arena> That's on the third of two calls to that method.
-
13:59:36: Found the problem! The
minimumSystemVersion
property inSUAppcastItem
was not being retained (or copied, to be precise about the correct behavior) by-setMinimumSystemVersion:
. So it wasn’t theSUAppcastItem
that was zombied; it was the item’sminimumSystemVersion
string.This also means that zac’s backtrace in #7543 is the same bug; the one in #7542 is missing a frame (for some reason).
Note how this was 21 minutes after I reproduced the crash on my laptop. This is why you must provide steps that can consistently reproduce your problem, if this is at all possible. Finding this bug would have taken at least an hour, probably more, if I hadn’t been able to reproduce it myself.
-
14:09: Posted second blog post, announcing that I found the bug and will release 1.1.1 “soon—if all goes well, within the hour”.
So much for that.
-
From the #adium IRC channel:
14:20:57: <Newtylicious> Did you really switch to geico? 14:21:08: <Newtylicious> and did it really take 15 minutes or less? 14:21:03: <Mac-arena> No. 14:21:16: <Mac-arena> Never let factual accuracy stand in the way of a good commercial reference.
-
From the #adium IRC channel:
14:21:53: <Mac-arena> I love flash memory 14:21:58: <Mac-arena> aka SneakerNet 3.0
-
From the #adium IRC channel:
14:23:42: <efge> Mac-arena: was it the same bug in 1.0.5 ? will people have to update manually from that version ? 14:24:02: <Mac-arena> efge: Well, that's a good question 14:24:12: <Mac-arena> I haven't confirmed yet whether the Sparkle in 1.0.5 has the same bug. 14:24:22: <Mac-arena> You raise a good point, and I thank you for it. We may be doing a 1.0.6 as well. 14:24:26: <Catfish_Man> it is likely 14:24:34: <Catfish_Man> iirc I wrote that code for a 1.0 beta 14:24:38: <Catfish_Man> possibly even a 1.0 alpha
-
14:42:02: Third and final debug build goes live. This one contains the fix for the crash, which was to make
-setMinimumSystemVersion:
copy its argument, then assign the copy to the instance variable. -
From the #adium IRC channel:
14:54:23: <vIkSiT> ah. so is the problem with sparkle itself, or the adium-sparkle integration? 14:54:31: <Mac-arena> Neither. 14:54:45: <Mac-arena> The official Sparkle doesn't have the problem. Only Sparkle ChatKit Edition and Sparkle Plus. 14:55:04: <Mac-arena> It was already fixed in the official Sparkle. 14:55:16: <Mac-arena> But that fix did not get carried over to S+, for some reason. 14:58:40: <jchang> is there a reason we didn't catch this bug during the betas? 14:58:49: <Catfish_Man> yes 14:59:00: <Catfish_Man>
minimumSystemVersion
was not enabled in the appcast correctly 15:01:02: <Mac-arena> Now it is, and we find out that it was broken in Sparkle+ :) 15:02:17: <Mac-arena> http://sparkle.andymatuschak.org/changeset/13 15:02:46: <Mac-arena> So, it was fixed 5 months ago and it never made its way to us :\ 15:02:54: <Mac-arena> (nor S+, which is how it would have gotten to us) -
From the #adium IRC channel:
15:06:52: <zac> can we temporarily fix the appcast now? :P 15:07:29: <jchang> zac: what do you mean? 15:07:51: <zac> removing the minimum requirement from the .xml will fix the bug 15:07:59: <zac> temporarily at least
I should have done this then.
-
15:20:24: Sent email to our development mailing-list explaining the situation and proposing a plan:
- Take the
minimumSystemVersion
elements out of the appcast. Since this would lead to Panther users downloading 1.1 again, also take out 1.1, leaving only 1.0.5 (with nominimumSystemVersion
element) in the appcast. - Push 1.0.6 with the fixed Sparkle+, for the benefit of Panther users.
- 24 hours later, release 1.1.1, put it in the appcast, and restore the
minimumSystemVersion
elements.
- Take the
-
15:37:46: “Michael” replies on the list suggesting that we only hold the 1.1.1 announcement from the appcast; 1.1.1 would be released everywhere else.
-
15:49:50: I reply to Michael agreeing with his plan, though honestly, I thought that that was what I’d originally said, and worded it accordingly. If you’re reading this, Michael, sorry about the mix-up. (The above list is taken directly from my original email to the development list. As you can see, I clearly wrote that we would hold the entire 1.1.1 release, not just the appcast. No, that doesn’t make sense to me either—Michael was patently right.)
-
From the #adium IRC channel:
16:45:25: <Maurits> hi, is the 1.1.1 version available from svn already?
-
17:15:25: No dissents on the development list, so I IM Chris (who’s just gotten back online) and ask him about the plan.
-
17:17:15: He asks basically the same question Michael did, and I (slightly puzzled at the time, but brushing it off) give basically the same answer I gave Michael. (To you, too, Chris, sorry about the mix-up.) He OKs the amended plan. I begin prepping two new releases.
-
17:21:03: I finally uncrashify the appcast as zac suggested (and as mentioned in my email), deleting 1.1 and the
minimumSystemVersion
elements. -
From the Adium development IRC channel:
19:02:29: <proton> you know peter what oyu really should do is make Adium 1.0.6 and Adium 1.1.1 point to a new sparkle feed that has minimum system requirements in it 19:02:38: <proton> and leave the current feed permanently pointing to 1.0.6 19:02:53: <proton> because there'll be people for quite a while with older versions that'll be going boom :) 19:04:31: <Mac-arena> Eh. I think most people will update within 24 hours. 19:04:56: <proton> i think that's deluded 19:05:19: <zac> 12.7% of our users currently use 1.0.4 still 19:05:24: <Mac-arena> Heh. 19:06:32: <Mac-arena> OK, will do 19:06:36: Mac-arena goes to amend the blog post 19:08:51: <Mac-arena> zac: Yo 19:08:54: <zac> yo 19:09:13: <Mac-arena> We need a new pair of feed URLs so we can abandon the old pair.
-
From the Adium development IRC channel:
19:19:34: <Mac-arena> What if we plan for future versions? 19:19:56: <zac> Mac-arena: hm? 19:19:59: <Mac-arena> update-enhanced.php?generation=2 19:20:07: <Mac-arena> This being the second generation of appcast feeds. 19:20:16: <Mac-arena> If anything like this happens again, we go to generation=3. 19:20:35: <Newtylicious> go forth and multiply, little appcasts 19:20:42: <Mac-arena> Each generation can represent a directory full of feeds. The current generation is what's updated by [the appcast-refresh script]. 19:21:07: <zac> Mac-arena: okay, do you want to add the argument "appCastGen=2" to the new builds? that's less hackish and cleaner 19:21:34: <zac> or just generation=2, whichever 19:21:51: <Mac-arena> Hmmm, yeah. We can even keep the same [appcast] URL that way.
-
About 19:14: Posted the third blog post, wherein I explained the day’s circumstances so far, the nature of the bug, and what we’d be doing about it.
-
From the Adium development IRC channel:
19:26:43: <Mac-arena> Gargh, this code is a mess 19:27:15: <Mac-arena> Our CodingStyle prohibits multiple returns, too. 19:27:37: <proton> sometimes ignoring the CodingStyle will make nicer code (and most of the time not) 19:28:06: <proton> hell, I've actually written code with a goto that didn't suck :) 19:28:13: <Mac-arena> proton: Same here. 19:27:44: <Mac-arena> This is one of the latter cases. 19:27:52: <Mac-arena> We're passed a mutable array 19:27:57: <Mac-arena> Expected to modify it 19:27:59: <Mac-arena> And guess what? 19:28:03: <Mac-arena> Two out of three times, *we don't*. 19:28:10: <Mac-arena> We return a new array instead. 19:29:03: <zac> Mac-arena: I'd say, let's just do the new appcast URL part now, and decide on new generational stuff later :) 19:29:13: <zac> It looks like it's more effort to do the latter now. 19:29:14: <Mac-arena> zac: No; I don't wanna go through this again 19:29:28: <zac> I don't want to break sparkle in 1.1.1 :) 19:29:31: <Mac-arena> It's hard to do *anything* to this code. This is no harder than anything else. 19:30:18: <Mac-arena> zac: This will use generation=2 19:30:46: <zac> k 19:34:59: <zac> k, my end works
-
From the Adium development IRC channel:
19:43:32: <zac> I can tell yo ureally quick if your generation stuff works 19:43:37: <zac> I just realized generation=2 isn't processed yet :)
-
From the Adium development IRC channel:
20:28:44: <Mac-arena> zac: Mind putting a bogus version in the gen=2 appcast? Like 1.42 or something? 20:29:01: Mac-arena is about to test his shiny new 1.0.6 and 1.1.1 20:28:59: <zac> sure 20:29:41: <zac> adium 1.9.7 is now out! 20:29:54: <Mac-arena> Woohoo!
-
About 20:38: I begin uploading 1.1.1 to my iDisk so that Augie can upload it to our CacheFly hosting.
-
20:52:59: 1.1.1 arrives on my iDisk. Immediately afterward, thanks to Transmit‘s Queue feature, 1.0.6 starts on the path to join it there.
-
21:46:37: 1.0.6 arrives on my iDisk.
-
22:12:08: Changelogs go up.
-
22:18:16: Replaced 1.0.5, added 1.1.1, and restored
minimumSystemVersion
elements in the appcasts in r20586. -
About 22:21: Appcasts updated on the website.
-
From the Adium development IRC channel:
22:24:12: <zac> Mac-arena: hmm, 1.1 isn't gonna see 1.1.1 under this plan 22:24:20: <zac> nor 1.1.2, 1.2, etc. 22:24:28: <Mac-arena> Hm. 22:24:51: <Mac-arena> It was a great plan, too. 22:24:52: <zac> 10.3.9 users are vastly the minority 22:24:57: <zac> should we inconvenience them [in favor of Tiger users, who are the majority]? 22:25:06: <Mac-arena> I'm OK with that. So what do we do? 22:25:08: <zac> put a "10.3.9 users: get 1.0.6 <here>" at the top of the 1.1.1 update 22:25:13: <Mac-arena> Sounds good. 22:25:18: <Mac-arena> In the changelog, you mean? 22:25:20: zac does so 22:26:24: <zac> 10.3.9 users: Get <a href="http://www.adiumx.com">1.0.6</a> to fix update checking. 22:26:26: <zac> ^ sound good? 22:26:35: <Mac-arena> Hm, no 22:26:47: <Mac-arena> 10.3.9 users: 1.1.1 will not run on your system. You need to get 1.0.6 instead. 22:26:49: <Mac-arena> And put it in h2. 22:27:54: <zac><p style="color: maroon; font-size: large;">10.3.9 users: Adium 1.1.1 will not run on your system. You need to get <a href="http://www.adiumx.com">Adium 1.0.6</a> instead.</p> 22:28:11: <Mac-arena> OK. How big is large? :) 22:28:25: <zac> about the same size as h2 22:28:35: <Mac-arena> Should work, then. 22:30:57: <Mac-arena> Let's change "10.3.9 users:" to "<b>10.3.9 users!</b>" 22:31:02: <Mac-arena> (note the exclamation mark, too)
-
About 22:26: Posted the release announcement for 1.0.6 and 1.1.1 to the blog.
-
From the Adium development IRC channel:
22:38:09: <Wengero> zac: (small little nitpick) #7542 in the changelog is linking to adiumx instead of trac.adiumx 22:38:27: <Wengero> so its getting a 404 22:38:37: <Mac-arena> Ah, relative links 22:38:41: <Mac-arena> That affects both changelogs, BTW. 22:38:46: <Mac-arena> And only the #7542 link. 22:39:13: <Mac-arena> <ul><li>Fixed <a class="closed ticket" href="/ticket/7542" title=""Check For Updates..." systematically crashes adium (closed)">#7542</a>, the Sparkle Plus bug that caused it to crash when passed a minimum OS version in an appcast (update feed).</li></ul> 22:39:35: <Mac-arena> Should be s#"/ticket#"http://trac.adiumx.com/ticket#
Finally, we were done. The bug was fixed and the fix was released.
So now you know the full details of how, in one day, we fixed two problems:
- Panther users being notified of a Tiger-only version of Adium
- All users crashing on update once we fixed #1
What a day. Apologies to everybody who had to put up with the crashing while we got that sorted out.
And no, it does not usually have that much drama. ;)
UPDATE 2007-08-15 08:30: Chris pointed out that he had suggested to Zac that Zac change his Adium defaults (prefs) to fake out Sparkle until the crash was fixed. I’ve added this to the appropriate point in the timeline.
Adium GSoC podcast!
Tuesday, July 10th, 2007This hasn’t gotten anywhere near the traction that I think it deserves, so I’m announcing it here too: We’ve done a podcast!
While we were supposed to be attending WWDC, Colin, David, and I played hooky and went down to the Googleplex to record an episode of the Google Summer of Code podcast. The podcast is hosted by Leslie Hawthorn, who runs the GSoC program. Adium participated last year and is participating again this year.
The episode, which is available from the GSoC blog post, is very funny and gives a bit of insight into how we run our open-source project.
It’s a little over 20 minutes of advice and programmer hilarity. Check it out. ☺
New utility: EasyMD5
Thursday, May 10th, 2007I’ve just released a simple application called EasyMD5. All it does is compute an MD5 hash for any file you drop on it.
I plan to use this to try debugging “your disk image doesn’t work” reports that we get on the Adium feedback list occasionally.
Ahhh, no more paging!
Thursday, April 26th, 2007My 4 GiB of memory from Other World Computing arrived today. Woo-hoo!
My Mac Pro came with 1 GiB of memory, which is the same as what I have in my Cube—but when you have four times as many processors, you need at least four times as much memory, especially for things like compiling programs. (Xcode will deploy one compiler per processor, and make can be instructed to deploy any number of tasks at a time with its -j flag.)
I waited to get more RAM because the Apple Store charged one-and-a-half kilobucks for the 4-GiB kit. Now, it’s down to only $1200—woo-hoo! But OWC still beats the Apple Store by more than half: the 4 GiB I bought cost me only $500. That’s still expensive by the Cube’s standards, but I was getting really tired of the paging, and it certainly beats the heck out of Apple’s prices.
Here’s the before-and-after of Adium build times:
1 GiB of RAM—2007-04-20:
svnversion %~/Projects/@otherpeoplesprojects/adium/trunk-clean(0) 19517 ___ time xcodebuild -configuration Deployment ** BUILD SUCCEEDED ** xcodebuild -configuration Deployment 252.57s user 232.52s system 196% cpu 4:07.33 total
5 GiB of RAM—2007-04-26:
svnversion %~/Projects/@otherpeoplesprojects/adium/trunk-clean(0) 19517 ___ time xcodebuild -configuration Deployment ** BUILD SUCCEEDED ** xcodebuild -configuration Deployment 243.32s user 220.05s system 232% cpu 3:19.21 total
Both tests were run with no existing build folder. Prior to each xcodebuild, I ran the same xcodebuild in order to defeat any caching issues.
New utility: qtsetclip
Wednesday, April 11th, 2007I’ve just released my latest command-line utility, qtsetclip. This is a utility that allows you to set the clipping region of a QuickTime movie to a rectangle you specify. I use it in producing the Adium screencasts, since Final Cut Express limits me to choosing among certain predefined resolutions. (Further details will appear on the Adium Blog on Saturday.)
LMX and Adium message history Q&A
Saturday, March 17th, 2007There’s been some discussion of LMX on the web since I announced LMX 1.0’s release. As I mentioned then, LMX is the library that powers Adium’s message history feature. Mostly, people have questioned whether XML was the best choice for logging given the message history requirement.
I recommend first reading my post on the Adium blog about message history. (If you came here from the Adium blog, sorry for the bouncing back and forth—that’s the last bounce, I promise. ;)
Welcome back. Let’s begin.
The questions and objections listed here are drawn from the comments on my LMX 1.0 announcement post, this article on the O’Reilly XML blog, the reddit post about LMX, and Tim Bray’s mention of LMX.
-
Why not just store the messages in reverse order? Then you wouldn’t need a backward parser; you could retrieve the n most recent messages from the top with an ordinary parser.
Because file I/O doesn’t have an insert mode; you can only overwrite. That means that Adium would have to rewrite the entire rest of the file every time it inserted a message (which is when the message is received or sent). That would get very expensive for long transcripts, and some Adium users leave their chats open all the time, so their transcripts would indeed get very long.
-
How do you append to the transcript? You must have to leave off the end tag, which means that the file is not a valid XML document until you close it, which would be bad if Adium crashed, since the end tag would never get written and the transcript would be broken XML.
Not so. This time, overwrite behavior is our friend: Adium simply overwrites the </chat> tag each time it writes a message, and appends a new </chat> tag in the same write. The file is always a valid XML document, thanks to overwriting.
Yes, this is slightly wasteful, but the waste here is constant (that is, it does not go up over time) and insignificant. The upsides vastly outweigh the downsides.
-
Why go with XML if you have to perpetrate such hackery as a backward parser? Why not use SQLite or a plain-text format?
SQLite: We would have had to include it with Adium, since Adium 1.0’s minimum requirement was OS X 10.3, and SQLite has only been bundled with Mac OS X since 10.4. LMX is much smaller than SQLite. Also, we’re not big on formats that aren’t directly human-readable.
Plain-text format: A simple format (e.g. TSV) would have some growing pains if we ever wanted to grow (or shrink) the format, and a more complex format would require a new parser from the ground-up just like XML does. For this purpose, we like XML’s trade-off between readability and extensibility, and LMX fills in the gap for reading from the end.
For more on formats we didn’t elect and why not, you can read our LogFormatIdeas page on the Trac (deprecated since we chose a format, but still around for posterity).
-
How will you determine the encoding of the data, or read entity declarations? Those things are at the start of the file, and you’re parsing from the end.
LMX naïvely assumes that the data is UTF-8 and that the application knows about any entities it will need. Yes, this is wrong, but Adium didn’t need anything different.
Either 2.0 or 3.0 will do a forward parse until the opening tag of the root element, in order to discover the actual encoding and any entity declarations. (I’m not doing it in a 1.0 version because 1.0’s parser is a hedge of thorns, and I’m not willing to touch it for something that most people won’t need anyway. And I’m tempted to leave this out of 2.0 as well, since 2.0 will be a big enough version with its rewrite of the parser in pure C.)
-
How does LMX tell whether –> is the end of a comment or simply an unescaped > following two hyphens?
Simple: It assumes it’s the end of a comment.
There’s no way to definitively find out one way or another without scanning all the way to the start of the data and backtracking. This is one of the pitfalls of a backward parser. It’s the nature of the game, so all I can do is say “make really sure you’re feeding good XML to the parser”. That includes not having unescaped ‘>’s in your text.
-
What about storing one message per line and scanning through the file line-by-line?
Because you can have a valid XML log file without that constraint, and constraints like that are the sort of detail you don’t want to rely on, because other apps can break them. (To Tim Bray: Part of the point of the Unified Logging Format is that we want other IM/chat clients to use it, which means that we should be forgiving when their output doesn’t exactly match ours.)
-
Can I grep these logs?
Mostly. You can grep an XML log in the usual way, but your expression can’t contain non-ASCII characters, <, >, or & unless you replace them with the appropriate entity references. We recommend using the search field in the Chat Transcript Viewer anyway.
I’m glad I finally announced LMX 1.0—not just because it is now, finally, out the door, but also because people have suggested new alternatives that we on the Adium team never thought of. For example, this reddit comment suggests saving one file per message (in a directory per chat), and this other one suggests inserting a fake start tag before the –nth message element, and the O’Reilly article suggests a hybrid XML+binary format. We never thought of any of these.
To be totally clear, we’re not switching—this post is a clarification, not an announcement. Two of those ideas won’t work for various reasons; the problem with the one-file-per-message idea can be overcome by tarring old chats. But LMX is not a future plan—we’ve written it and it’s here, and the same goes for the Unified Logging Format.
Call it inertia, but replacing either one with something else will require either the existing solution to break or the proposed replacement to exhibit massive, world-changing superiority. These things are done and they work, so at this point, we’re not going to rock the boat. It ain’t broke anymore, so we’re not fixing it.
LMX 1.0 released
Saturday, March 3rd, 2007Some of you know that I’m a developer on Adium. (Hopefully all of you; it is mentioned in the sidebar. ;)
Adium has a feature called “message history”. When you open a new chat with a person, message history shows you the last n messages from your previous chat with that person. Since 1.0 (which changed message history to draw from the logs rather than separate storage and changed the log format to be XML rather than bastardized HTML—more info on the Adium blog post), message history has been implemented using a library that I wrote called LMX.
LMX is a reverse XML parser. Whereas most XML parsers (AFAIK, all of them except LMX) parse the XML data from the start to the end, LMX parses it from the end to the start. Thus, while characters are kept in their original order (“foo” will still be “foo”; it will not become “oof”), everything else is reported in the reverse order: elements close before they are opened, and appear from last to first. All this is by design, so that Adium can retrieve the last n message
elements without having to parse all the message
elements before them.
Today, LMX gets its very own webpage (not just a page on the Adium wiki, but a real webpage), and is released at version 1.0. It’s the same code as shipped with Adium 1.0.1, but shined up into a release tarball.
So, if you too ever find yourself in desperate need of a reverse XML parser, now there is one.
The fun of length-declared strings
Thursday, November 16th, 2006Leak: 0x3cf9d1e0 size=160 string 'h>10:47<br /></span>Tuesday, November 7, 2006<br /><br /><br /><br /><br /><br /><br /><br /><span style='
Searching the log of LMXTestApp parsing the log file: No occurrences of “h>”.
Searching the log file itself: No occurrences of “h>”.
*scratches head*
python %~/Projects/@otherpeoplesprojects/adium(0) >>> len('>10:47<br /></span>Tuesday, November 7, 2006<br /><br /><br /><br /><br /><br /><br /><br /><span style=') 104 >>> ord('h') 104
Aha!
The light-bulb image above is a cropped and scaled version of this photo, which is under a Creative Commons Attribution Non-Commercial 2.0 license.
How to apply changes in a framework to a versioned copy of it
Tuesday, November 14th, 2006In Adium 1.0, we’ll have a number of external frameworks:
- FriBidi.framework
- Growl-WithInstaller.framework
- LMX.framework
- OTR.framework
- Sparkle.framework (actually SparklePlus)
So whenever any of these frameworks changes — especially if it fixes a bug that we’ve been suffering from — we build the framework in our copy of their source code, and then we drop the new built framework into our source repo and commit it.
But I often see it done this way:
- svn rm Foo.framework
- svn ci Foo.framework
- cp -R ../../Foo/build/Release/Foo.framework Foo.framework
- svn add Foo.framework
- svn ci Foo.framework
This is wrong, for three reasons:
- Commit history for the framework is lost. svn doesn’t relate the new framework to the old framework that previously occupied the same path.
- Much space and bandwidth is wasted as all of the files in the framework (including nibs, images, plists) are copied yet again, even those that were not changed.
- It takes two commits to do one job.
Here’s a better way.
- pushd ../../Foo/build/Release
- tar cf Foo-framework.tar Foo.framework
- mv Foo-framework.tar $OLDPWD
- popd
- tar xf Foo-framework.tar && rm Foo-framework.tar
- svn ci Foo.framework
tar will overwrite every file in the framework, and of course any new ones will be created. A quick svn st will show you which files are new (unversioned) or modified; if there are any unversioned files, you should of course svn add them before committing. And then when you commit, the commit history of all the files is preserved, so you can do svn log (or the equivalent operation in Trac) later, and svn will apply its own smarts about exactly what should be copied.
The only downside to this approach is that any files that no longer exist in the old framework won’t be removed. If you have reason to believe that any files were deleted, you should perform ls -R on both directories, diff -u the two results, and look for “-” lines. But most of the time, this isn’t necessary; how often does anything ever get removed from a framework?
So, anyway, please use tar, for the sake of the commit history. ☺
UPDATE 11:55: If you want an example, here’s Adium changeset number 18243, in which I did this to the LMX framework. See how small that changeset is?
UPDATE 14:08: Fixed SparklePlus link.
Observations on the Mac Pro
Thursday, October 26th, 2006- Wow, this thing is big. So much larger than my Cube.
- I need a DVI-to-ADC adapter to use my ADC-based monitor. Suck.
- The Cube effect is yummy.
- Finally, I get to play all these games and watch all these videos without asking to use mom’s iBook!
- It’s nice to look at my CPU Usage (not-yet-finished 0.4) and see as many as three cores (did I mention that it’s a quad?) running at 0%.
- I no longer need hubs. I no longer need the external USB speakers (with headphone jack) nor iMic, freeing up two USB ports (exactly how many my Cube had). So now I can plug my keyboard into the first, my monitor into the second, and my printer into the third. If I still need more ports (e.g. flash memory), I have two on the front.
- Speaking of which, I no longer need my FireWire repeater either. I can plug my iPod into the front FireWire port.
- The headphones also plug into the front, but they jut out too far for my comfort. I’ll buy an L connector at Radio Shack to avoid me shearing off the headphone connector in the jack.
- The drive bay covers slide down, rather than flapping open like a drawbridge. That’s cool.
- Computer name: “Four-Sided Cheese Grater”. (Did I mention that it’s a quad?)
- Rosetta works well. ShadowIRC (PPC-only) doesn’t seem any slower on this machine than it did on my Cube.
- Apps that behaved slowly on my Cube (e.g. QS) are quite fast on this one.
- OmniWeb actually launches fairly quickly on this machine. Not like Safari, though: half a bounce.
- I look forward to doing a make -j4 (or maybe more).
- As Colin points out, I have gone from last to first in terms of Adium committer developer processing power. (UPDATE 05:57: Andreas Monitzer, one of our GSoC students, has a 3 GHz four-core Mac Pro; mine is only 2.66 GHz. Rats.)
- I now have a much better reason for recompiling my various command-line utilities, as I tend to do whenever moving to a new machine. This time, I’m changing architectures.
- The Mac Pro is wider than my Cube, making it a better platform for my headphones (each ear pivots, so I can stand it up on them rather than having to hang or lie the headphones somewhere).
- In the box, there’s a 3″ DVI–DVI cable. One end is a female DVI connector; the other end is a male DVI connector. I see no difference in pinouts; therefore, I am at a loss to explain what this cable is for.
- There’s no Eject button on the computer; you’re supposed to use the one on the keyboard for that, but I’d rather not unplug my perfectly-good iKey just to get an Eject button. So I’ll just have to add the Eject menu extra to my menu bar.
- No more convection cooling. ☹
- Startup time (from *bong* to loginwindow): 18 seconds. This compares to 45 seconds for the Cube. The progress bar goes from 0 to 100% in less than a second.
Woo-hoo, T-shirt!
Tuesday, October 3rd, 2006ISO 8601 parser, version 0.4
Thursday, August 31st, 2006I’ve released version 0.4 of my ISO 8601 parser and unparser.
Colin noticed that 0.3 didn’t work so well for parsing the datestamp in the filename on an Adium log file (those datestamps being in ISO 8601 format). You see, Adium generates the filename with a ‘.’ instead of a ‘:’, since ‘:’ is special on HFS/HFS+. 0.3 expects a ‘:’ and only a ‘:’.
So I added new methods that take a time-separator character. Now you can pass ‘.’ (or anything else, except NUL) when you need to. The old methods are still around, in case you don’t. This is the case, for example, for datestamps inside a chatlog file, as opposed to in its filename.
ISO 8601 unparser, version 0.3
Tuesday, June 6th, 2006Colin found some bugs in my ISO 8601 unparser when he added it to Adium. I’ve fixed these (as he has in the Adium copy) and released it as 0.3.
The parser is still unchanged.
Technorati tags: ISO 8601.
ISO 8601 unparser
Thursday, June 1st, 2006The unparser is finished. I’ve bundled it with the parser (which hasn’t changed since 0.1) and released it as 0.2. Check it out.
I was working on my own algorithm for computing the week date, but couldn’t get it to work correctly on every year, so I finally settled for implementing Rick McCarty’s week date algorithm.
Technorati tags: ISO 8601.