Archive for the '@Uncategorized' Category

How to QA

Monday, July 3rd, 2017

In my day job, I’m a QA (quality assurance) engineer for a Large Software Company. Today I’ll tell you how to prepare yourself to do a job like mine.

Fair warning: “QA” is a big tent; the details of a QA job vary widely, and your QA job may be very different from mine.

File bugs

(Ira Glass voice) File a lot of bugs. File a huge volume of bugs.

The ultimate responsibility of a QA team overall, and of QA engineers individually, is to produce actionable bug reports that can and do get fixed. The details vary widely, but everything comes back to that.

A good bug report says:

  • what you did
  • what you expected to happen
  • what actually happened, and how that differed

That’s the minimum, actually. That’s the least that’s needed for a bug report to be actionable.

Ideally, a bug report should also include needed diagnostic info (such as a sysdiagnose on Apple platforms), screenshots/video if applicable—as much info as possible, at least at first, to be sorted through for the gems of info that illuminate the actual problem. As a QA engineer, providing as much of this info you can gather is Actually Your Job.

Ah, but what to file? If you’re QAing something manually, you’re looking for:

  • any sort of friction (possible interface design issues)
  • any sort of fault (crashes, hangs, data loss)
  • anything in between (anything that did not do what you expected it to do)
  • anything you don’t know what you expect it to do (can be a design/empathy-for-the-user issue)

Also test anything that had been broken before. Look for regressions (previous problem returned) or new problems (“OK, you fixed X, but now it has problem Y”).

Practice, practice, practice

For filing formal bug reports, two good ways are filing Apple developer bugs and filing bugs with open-source projects.

A few caveats re open-source:

  • Many open-source projects need fixes more than they need more bug reports, so don’t be surprised if folks don’t rush to thank you for adding to the pile. Concentrate on high-value bugs (security vulnerabilities, crashes, providing desperately-needed steps to reproduce) rather than nitpicks. (Making this judgment call is itself something that’s valuable to practice.)
  • Search for duplicates first. Apple actually generally likes duplicates and uses them to influence triage and prioritization decisions, but open-source projects may resent the extra scut work of duping bugs together. (Finding existing bug reports is an underrated skill that is also valuable to practice.)
  • I’m a cishet white guy. Your mileage may vary a lot regarding how you and your contributions are received if you are not—anything from well-intentioned over-helpfulness (mansplaining, assumed noviceness) to outright misogyny/racism/anti-trans assholery.

Some open-source projects do code review and/or API review, which can be good practice for spotting designs that invite bugs (random example: “you’re taking a pointer but not the size of the buffer, so the API can’t check that it won’t go out of bounds”).

In that sort of setting, practice asking questions. These could be clarifying (“what does it mean if this returns nil/0?” “This is typed as a String. Are there constants, or what sort of strings should folks pass here?”) or more Socratic (“what does this API do if I pass a pointer to a buffer that’s too small? how does it detect that?”). Even if the immediate response is “that can’t happen” or “don’t do that”, established members of the community/team may back up your question with pressure to resolve the issue (“this should take a length with the buffer, or better yet return a Data”).

Lastly, volunteer for beta tests. I’ve beta-tested Flying Meat’s Acorn once or twice, and I think Panic have also solicited beta testers in the past. Beta-testing is extremely good practice for exploring an in-development product looking for friction and faults and writing up your findings.

As a beta tester, write up everything. Don’t be shy—if it looks wrong, write it up. I have sent in multiple pages of issues and you know what? That makes you worth your weight in gold. (90% of volunteer beta testers are just there for the possibility of a free license. Any developer who’s running a beta test program wants bug reports. They want them urgently—preferably while the product is still in beta.) Even if half the stuff on my list is stuff they meant to do, it’s still worth a second look if a beta tester (or more than one) objects to it, and the other half of the stuff on my list is stuff they may not want to ship with.

The QA mindset

Use your imagination. Try things you wouldn’t ordinarily. Follow the “what does that do?” impulse. Find the cold paths.

Everything that the product engineers thought of is probably really well-tested, and any issues that remain have turned into invisible corners. Your job is to remind them of the invisible corners and tell them about the things they haven’t encountered at all.

Question everything. Is that design optimal? Is the UI copy clear? Could a new user understand both? Was that actually the correct result? Was it the best result?

If you find yourself asking “What does that do?”, that can be a sign of inaccessibility to novices. Inversely, you should learn the product’s domain so you can spot things that don’t make sense in the domain. Try to develop the ability to do both: know the product’s domain and simultaneously spot barriers in the product to users entering that domain.

Be an advocate for users

Make sure the interface is accessible (overly-similar names can cause problems for dyslexic folks; reliance on color or images can cause problems for color-blind or visually-impaired folks).

Familiarize yourself with accessibility tools (on the Mac: Accessibility Inspector, everything in the Accessibility control panel, SimDaltonism) and use them on the product, and file bugs when you get stymied.

Challenge ableist/misogynistic/gender-binarist/racist/otherwise-problematic copy or artwork, using the dual justifications of “this is the right thing to do” and “not doing this is going to repel users/customers and/or cause PR trouble”.

Your QA job may vary

As I mentioned above, “QA” is a big tent. What I’ve described is an important baseline mindset and skillset to a QA engineer, but major portions of an individual QA job could be more sysadmin-oriented (e.g., administering an automated testing or CI fleet) and/or include work with specialized, company-specific internal tools that you’ll have no way to practice the use of outside of that department.

Know at least one scripting language such as Python or Ruby, whether for use in your own QA work (e.g., generating test data, filtering logs) or for working on automation systems (e.g., maintaining Python or JavaScript scripts that drive UI tests). If you’re starting from scratch (and absent any particular job-specific requirements), I’d recommend Python, which you can pick up in a few months, and from which you can learn similar languages like Ruby afterward.

Especially as more automated QA (unit testing, UI automation) becomes more the norm, I’d strongly recommend developing your system administration skills on any platforms you may end up working on. It’s been awhile since I had to start from scratch on this, so I invite you to suggest resources in the comments for learning system administration (on the Mac or otherwise) in 2017.

Chess variant idea(s)

Saturday, May 13th, 2017

Each player has one spy among their opponent’s pieces.

On your own turn, you can switch out one of your opponent’s pieces for an equivalent piece in your color (e.g., black rook for white rook), then move it as one of your own. Ever after, it is one of your pieces.

Possible variants:

  • Reveal your spy on your opponent’s turn instead of your own (you take their turn instead of them taking it).
  • Reveal your spy on any turn.
  • More than two spies.
  • Unlimited spies.
  • Double agents (after revealing your spy, your opponent can reveal that it was their own spy, taking the piece back, ending your control of it).
  • Arbitrary-multiplier agents (revealing a double agent does not end control: a N-ple agent can always be revealed to be an N+1-ple agent).
  • A requirement that you choose your spy beforehand and seal the knowledge in an envelope (rather than choosing the spy to “reveal” at any time). Open the envelope when you reveal your spy.

I don’t know enough about chess to be able to predict how well this would work, or whether anyone has thought of this before (it’s quite likely).

Super simple solder spool spindle

Monday, November 14th, 2016

(Also works for spools of wire.)

  • One ³⁄₈″-inner-diameter galvanized pipe cap
  • One ³⁄₈″-inner-diameter 4″- or 5″-long galvanized pipe nipple
  • One ³⁄₈″-inner-diameter to ¹⁄₂″-inner-diameter galvanized pipe reducer
  • One ¹⁄₂″-inner-diameter coupler
  • One ¹⁄₂″-inner-diameter floor flange

Screw them all together, in the reverse order, putting the cap on last after you put the spools onto the pipe.

Photo of my solder spindle, along with a prototype of it holding some wire spools.
Left: My solder spindle built from the above recipe. This is with the 5″ pipe nipple.
Right: My wire spindle, built using ¹⁄₂″-inner-diameter all the way down, which it turns out is too big for one of the solder spools. (It was the first draft; the final spindle on the left is the second iteration.)

Normal

Monday, September 12th, 2016

I just got this ad for Jane Kim, who’s running for State Senate here in California:

“JOIN US as we work to lift more families into the middle class through better public education”! and “Median Annual Wage for City College Graduates: $59,800. Median Annual Wage for High School Graduates: $48,700”

Leaving aside the matters of whether those are livable wages in the City, particularly after taxes and any available savings contributions, and especially for “families” as stated in the headline…

“Join us as we work to lift more families into the middle class…”

A “middle class” necessarily implies a lower class and a higher class.

Right off the bat, this proposition does not even consider lifting families into the upper class. Just the middle class—that’ll do for now, right?

Lifting “more families into the middle class” implies not eliminating the middle class—there’ll still be one, which means there’ll still be a lower class, which means there’ll still be people/families in it.

Certainly this proposition does not suggest ending classes altogether—no; there’ll still be classes, at least three of them. And some will be in the upper class and some will be in the lower class and hopefully more of the latter will move up… to the middle.

Oh, OK.

I feel like this is a lukewarm dish. That’s all you’ve got to offer? You want to make this not only a headline on your flyer, but literally the entire topic of an entire flyer?

I’m not even saying it’s a bad idea. Just… it lacks ambition. It feels resigned. This is the best we can do, it implies.

Really?

NB: I feel I should make explicit that I’m not against Kim necessarily; I haven’t looked at any other candidates. And maybe she’s got lots of good policies and/or ideas. Just, this one ad caught my eye and got under my skin.

More on the absurdly small size of storage

Thursday, February 3rd, 2011

According to Wikipedia, a drop of water can be up to 6 mm in diameter. That works out to 0.1131 ml.

The volume of a microSD card is 0.165 ml.

The largest size of microSDHC card is 32 GB.

This means that just over 21.9 GB of data will fit in the space of a drop of water.

My new movie-watching mode

Wednesday, March 31st, 2010

The DVD drives in the last few years’ Mac models are quite loud. When watching a movie from a DVD, it sounds like I have a very-high-speed fan only a few feet from me, only without the cool breeze.

This is a problem because I keep my sound volume cranked way down (to the benefit of my hearing), so the DVD drive effectively drowns out the movie. I don’t have this problem when watching a video from a hard drive or the internet.

So here’s what I do:

  1. Copy (straight across—no decrypting) the DVD to my media hard drive on my desktop machine.
  2. Eject the DVD, put it back in the case, and put the encased DVD away.
  3. Watch the movie in VLC.

Yesterday, I successfully tried a new variation on this procedure:

  1. Copy the DVD to my media drive on my desktop machine.
  2. Eject the DVD and put it away.
  3. Make the Movies folder on the media drive a shared folder.
  4. With the desktop machine downloading stuff from the internet or maybe seeding a (legal) torrent, go on my laptop in another room and mount the desktop’s Movies folder on the laptop.
  5. Watch the movie (in the mounted shared Movies folder) in VLC.

You’ll notice that I did not copy the movie to the laptop. I opened the copy on the mounted local share, so VLC on my laptop was effectively streaming the movie from my desktop.

This requires a bit of tweaking in VLC’s Advanced Preferences. The default settings waited too long to read more data from the “disk”, so the movie was jerky. I fixed this by appending a couple of zeroes to the latency fields for the three relevant “access modules”: DVD without menus, DVD with menus, and file. (You may only need to set the last one; it didn’t work right until I set that one, and once it did, I didn’t do any further investigation.)

Once I’d made those small changes, the movie streamed fine over the local network.

My Amadeus Pro sonogram preset

Monday, March 15th, 2010

When I do audio editing in Amadeus Pro, I find its Sonogram command useful. That command gives me a three-dimensional graph of frequency distribution through time:

Where x = time, y = frequency, and z = amplitude.

The default setting shows the z axis with a color gradient of white through orange through blue to cyan, which I found hard to read. What you see above is my variation on the built-in “Greyscales” preset: Just like a printout, white is dead, black is maxed.

The difference with my preset is that the original is linear, whereas mine is a curve. Here’s the preset editor, showing my preset:

The “Grayscale Nonlinear” preset.

Sampling the swatches in that screenshot will tell you that the five equally-spaced gradient stops are:

  1. White
  2. 50% gray
  3. 25% gray (at the halfway point)
  4. 12.5% gray
  5. Black

I’ve found that these settings make the spectrogram clearly readable without my having to fiddle with the “range” and “gain” sliders.

The focus switch on the Kodak Zi6

Wednesday, December 23rd, 2009

The Kodak Zi6 pocket video camera has a switch on its upper-right corner that selects its focal length. It appears to have two settings: Distance, indicated by mountains, and macro, indicated by a flower.

That appearance is wrong. The switch is not binary; it actually selects within a range.

This is useful for things that are close to the camera, but not macro close. For an example, I took pictures of my keyboard. Fully macro was unusably blurry; fully distance looked like this:

Slightly blurry.

whereas moving the switch a little bit up from distance mode got me this:

Perfectly clear.

Neither photo has been adjusted at all. Both photos were taken freehand, but I can vouch for their sharpness-accuracy: What you see in the first photo is focal blur, not motion blur.

So, if you have a Kodak Zi6 (or, perhaps, one of their other cameras, especially in the same family) and you want to take a photo or video of something close but not macro close, adjust that focus switch a little bit.

Andy Richter Controls the Jeopardy! Board

Thursday, September 17th, 2009

Every once in awhile, Jeopardy! does a special series of episodes called Celebrity Jeopardy!, where the contestants are celebrities playing for charity. Most of them are laughably bad, and I mean that literally: I (and the studio audience, and the contestants themselves) actually laugh at their poor responses. The celebrities are mainly there to provide entertainment, and to bring attention to their charities and their own projects and to Jeopardy! itself; playing a few rounds of Jeopardy! is a formality.

The problem is simple: Regular Jeopardy! games are populated by contestants who pass one or more screening tests that verify that they meet some minimum standard of knowledge, comprehension of English, and ability to operate the buzzer. Celebrity Jeopardy!, on the other hand, is populated by celebrities. They have no qualifications other than being famous and being willing to be on the show to benefit a charity.

This year, they’re doing a variation of Celebrity Jeopardy! called the “Jeopardy! Million-Dollar Celebrity Invitational”, where they have one Celebrity Jeopardy! episode every month for nine months, followed by a three-day semifinal and two-day final in the tenth month. The other episodes in each month are at least mostly regular episodes, perhaps with a Tournament of Champions, Kids’ Tournament, or College Championship at some point.

Tonight was the first episode in this celebrity tournament. The contestants were Andy Richter (for St. Jude Children’s Research Hospital), Dana Delany (for the Scleroderma Research Foundation), and Wolf Blitzer (for the American Cancer Society). Richter’s performance tonight inspired this post.

Andy Richter decimated his opponents. Dana Delany did pretty averagely, entering Final Jeopardy! with $2800; Wolf Blitzer ended up well negative (they gave him $1000 with which to play in Final anyway, since this was a charity event). Richter got most of the answers, and he almost always gave right questions. He even ran a category.

This tournament, every celebrity’s charity is guaranteed a minimum $25,000 donation, and each winning celebrity’s charity is guaranteed a minimum $50,000.

Andy Richter won $68,000.

It makes me think that most celebrity contestants just show up, figuring that it’s $25k their charity wouldn’t have otherwise, and all they have to do is stand there for an hour or so and throw out an occasional right answer—something regular contestants can’t do, because they have to earn their spot behind a podium. And it makes me think that Andy Richter didn’t do this: He took this tournament seriously, and put in the exact same study and buzzer training that any regular contestant would have done.

I could be wrong. Maybe the other two contestants just got screwed on the buzzer (although that wouldn’t explain Wolf Blitzer’s score). But if I’m right, Richter’s charity should be really happy with the work he put into it.

I have one idea for how Sony (the company that runs Jeopardy! in the US nowadays) could improve things. Tell every celebrity who agrees to play that they will have to take the same screening test as a regular contestant would have to. If they fail, no problem: They still get to play. But if they pass, Sony doubles its minimum donation guarantees for their charity. That may be enough of an incentive for celebrities to take it as seriously as Richter seems to have done, and bring a real game to the Jeopardy! set like he did.

You might object that I shouldn’t complain about these celebrities who are playing for charity; after all, isn’t it great that the charities get all these thousands of dollars for free? True, but the celebrities can win even more money for them, and I think Richter demonstrated that it’s much more entertaining when they do.

On a related note: I wish Jeopardy! episodes were available online.

UPDATE 2009-09-19: YouTube to the rescue: Part 1, part 2. Thanks to Ian Baird for linking to one of them.

Popularization

Monday, May 11th, 2009

Here’s something I found interesting.

If you’re on Twitter, you might have noticed a trending tag today, #TMItweets. People have been deliberately posting tweets with Too Much Information, tagged with #TMItweets.

The interesting part is who started it.

If you go all the way back on the Twitter search for the tag, you’ll find that the earliest tweet in the search engine’s short memory is this one from Alex Fayle, 20 days ago. I don’t think he’s who started the current trend.

The next few tweets are. They are, in order:

  1. Tai (her tweet)
  2. Dora Bianchi (her tweet)
  3. Hannelore Ellicott-Chatham (her tweet)
  4. Raven Pritchard (her tweet)
  5. Pintsize* (his tweet)
  6. Penelope Gaines (her first tweet with the tag; her second)
  7. Faye Whitaker (her tweet)

If you read Questionable Content, you recognize those names. These are the official QC character accounts, they each have thousands of followers, and they’re all written by Jeph Jacques.

So, basically, this trend started among a closed circle of fictional characters, and migrated into the real world from there within minutes**. I can’t think of a medium where this could have happened before (without including it in the primary medium—in this case, in the comic strip).

Like I said, I found it interesting.


* Not linking directly to Pintsize’s account page because the links he posts are almost all NSFW. Consider yourself warned.

** Pintsize posted his #TMItweets entry at 5:05:46 UTC, and the first subsequent non-QC entry came at 5:07:53 UTC. I used the Twitter API show-status method through curl to obtain these timestamps.

Scale

Wednesday, April 29th, 2009

264 doesn’t sound much bigger than 232, does it?

264 is 232 × 232, but this relation is hard to put into perspective. Even saying that 264 is more than four billion times as big as 232 doesn’t adequately convey how much larger it really is.

Let me do that for you now.

I wrote a program that counts from 0 to 232. It takes about 11 seconds to run:

% ./count-up
2009-04-29 09:52:55.387 count-up[57932:10b] Time to count up to 4294967295: 11.313282

Now, as I said, 264 is 232 × 232, so the time to count to 264 is likewise the time to count to 232 (11.313282 seconds) multiplied by 232. That works out to:

  • About 48,590,176,200 seconds
  • About 13,497,271 hours
  • About 562,386 days
  • About 1540 years

Simply by doubling the exponent, we increase the time it would take my late-2006 four-core Mac Pro to count up to the number from 11 seconds to a millenium and a half.

Storage

Friday, March 27th, 2009

The Apple ProFile hard drive held 5 or 10 megabytes in 11.15×43.89×22.38cm = 10952.17893 cubic centimeters.

A microSD card is 1.5×1.1×0.1cm = 0.165 cubic centimeters and can hold up to 16 gigabytes.

If you stacked up enough microSD cards to fill (as closely as possible*) the volume of a ProFile, you would have 29×20×111=64380 cards totaling 1.03008 petabytes (1030.08 terabytes)—206,016,000 times the capacity of the hard drive you’d displaced (assuming the 5 MB version).

Also, the ProFile cost $3499 in 1981 (about $8194 in 2008 dollars), whereas a 16 GB microSD card costs about $70 as of March 2009. If you somehow bought enough 5 MB ProFiles to make up the difference in capacity (3200 of them), at the original MSRP, you would spend $11,196,800 (about $26,222,014 in 2008 dollars).

(I used Robert Sahr’s inflation conversion-factor tables to convert to 1981 dollars to 2008 dollars.)


* It’s possible that you could fill out the space a bit more by standing some cards on edge, but I don’t feel like doing that much multiplication.

Apple Bug Friday! 75

Friday, March 21st, 2008

This bug is TN2083 doesn’t document all known Session types. It was filed on 2008-01-19 at 16:44 PST.

(more…)

The Sinister Divide

Sunday, March 16th, 2008

In Dvorak, the Sinister Divide is between y and f, between i and d, between k and x, and between 6 and 7.
In QWERTY, the Sinister Divide is between t and y, between g and h, between v and b, and between 6 and 7.

Dear Lazyweb: A Frank Caliendo question

Friday, February 15th, 2008

You may have seen Frank Caliendo’s recent Dish Network commercials, in which he does pitch-perfect impressions of several celebrities. Two of the three I’ve seen are George W. Bush and John Madden.

But, and this is my question, who is this supposed to be?

UPDATE 2008-02-17: I just watched it again, and I think Ian is right—it’s Al Pacino. Thanks!

A teleport tip

Friday, February 1st, 2008

If you have multiple Macs, I recommend that you install teleport.

teleport's preference pane lets you lay out your Macs the same way the Displays preference pane lets you lay out multiple displays.

teleport is a software KVM: Simply move your mouse cursor through the edge of your computer’s screen into the other computer’s screen, just as if it were another monitor connected to your Mac. From then on, your mouse and keyboard control the other Mac, until you move the mouse cursor back through the edge, back to the Mac whence it came.

Here’s the tip: When you first set up teleport, it’s tempting to turn on the “Switch with a delay” option, on the theory that it will help prevent you from switching accidentally. It doesn’t. Turn that off. It’s a good theory, but all it does in practice is make it harder to correct an accidental switch, not to mention harder to switch on purpose.

Leave that option off, and it will be as easy to undo an accidental switch as it is to commit one.

My 2008 public key

Tuesday, January 1st, 2008

As usual, I’ve created a new public key for 2008. The old one has expired. The new key’s key ID is A54AE2A6, and its fingerprint is:

3976 D719 9791 B921 FCD3  CCA7 99CE E202 A54A E2A6

Also as usual, you can obtain my key from pgp.mit.edu (or, probably, most reputable keyservers) or my public-key file.

Google sure is fast nowadays

Wednesday, December 26th, 2007

I created a page on the Adium wiki at 07:07 PST, and Google had it indexed by 08:01. This screenshot is of a search for “darcs patch bundle binary”.

The times shown in the screenshot are in EST; I created the page at 07:07 PST, and found it among those search results at 08:00 PST.

I think somebody mixed up their phishing emails

Sunday, December 2nd, 2007

From: Bank Of America Security Dept. Subject: IMPORTANT: Security Issues [Incident: 0409252]. Body: You have just received a virtual postcard from a family member!
What is this, scamming for victims with ADD?

I cannot leap tall buildings in a single bound

Wednesday, November 28th, 2007

And despite Google Transit’s insistence, I cannot legally cross all those lawns:

Google Transit told me that to get from my location in Cupertino to 1 Infinite Loop, I can simply walk in a straight line (which it depicts as an arc)—crossing through a residential block and apparently jumping over most of the Apple Campus.