July 25, 2014

Don't lose sight of the problem you're trying to solve

Here are a couple of things which have reduced me to tears (a mixture of laughter and exasperation) during the past few days. Both seem to stem from well-executed solutions to the wrong problem.

Firstly, from the world of "next-generation" games consoles. All of the next-gen contenders (Microsoft's Xbox One, Sony's Playstation 4, and Nintendo's Wii U) were criticised at launch for their excessive load times; many consumers, when confronted with a 30-second boot screen, thought that their consoles had frozen or were defective.

Microsoft's answer? A new boot screen, with dots to indicate loading progress.

Nintendo's answer? Reducing the damn boot time.

Both of these solve the problem of "consumers thinking that our consoles have crashed", but only one of the solutions also delivers an appreciable customer benefit. Did you spot which one it was?

My second exasperation this week was when using Google Docs within Firefox. I've tried to trace the root cause; it appears to be something to do with browsers isolating themselves from the clipboard (the problem does not affect Chrome), but whatever the problem, in what crazy world is this the best solution?

The Edit > Copy menu in Google Docs

I've known testers who've been guilty of similar. On one occasion, I asked a colleague to walk me through a relatively innocuous bug that he'd logged, where some of his feature's settings were being reset to their defaults when starting the application. He hadn't written-down the reproduction steps (thanks for that...) but told me that it wasn't a problem as he could clearly remember the steps.

  1. Open a project.
  2. Change SettingX from 1 to 2.
  3. Press F6; the application crashes.
  4. Reload the application, and note that SettingX is set back to 1.

"Woah," I said, "Why are you obsessing over step 4? What the hell happened in step 3?"

It turned out that there was a previously-unreported regression whereby one of our interfaces (launched via F6) would crash in a particular scenario. But because our tester was fixated on testing one specific feature, he excluded any information which seemed to be outside of that scope. (It was a pretty common mindset for this particular team, and needless to say, that's one of the reasons why I no longer work there.)

When testing, there are key questions that should always be at the forefront of our minds. These include "When do we stop testing?" (Michael Bolton), "Who are my clients?" and "Why am I even doing this?" (Anand Ramdeo). If you lose sight of these pillars of testing, then (to quote from South Park) you're gonna have a bad time.

  • LinkedIn
  • Tumblr
  • Reddit
  • Google+
  • Pinterest
  • Pocket
Comments powered by Disqus