Version Support Hell

Posted Oct 8th, 2015

Facebook. It’s a love-hate relationship. I love seeing my friends and family from afar, what they are up to, how their children are growing. I hate how ingrained it is in everyday life. How on earth did we manage when I was growing up? (Snail mail, actually sending photos that you had to have developed; heaven forbid you pick up a phone and talk to someone). The most amusing aspect to me is how much complaining I see in my news feed every time facebook pushes a change: “No, I do not want to be added to a group without my permission, thank you very much,” or “Why is my grandmother allowed to tag random people I don’t know on my photos now?” (That might be a question I asked. She may need to take a few lessons in what tagging is actually for; I might need to redo all my privacy settings).

In my experience, nobody likes change. It takes us out of our comfort zone. Take it or leave it though — Facebook is on to something. They don’t have to deal with running several versions of their product! They deploy a change, people moan and groan about it, and then get used to it.

In my world, it’s not so simple — a world where we have self-hosted clients (who can determine what version, to an extent, they want to be on) and clients in the cloud (the latest release). We have to test everything. I long for the day where all clients are on one version, and I no longer have to worry about supported upgrade paths, or testing a bug across three releases.

So. Much. Testing.

Let’s say you find a bug. If you have more than one version out in the world for clients, you probably need to see if that bug happens on any release you say you support. Did you decide to fix that bug? Guess where it needs to be tested. No, not just the latest. Do you know how much fun that is to track? (It’s not.)

Do you support certain operating systems (server and/or client), browsers, mobile devices (and mobile operating systems), accessibility, languages? Your list of tests just grew, perhaps exponentially. Honestly, when I see issues come my way that could apply to any of our releases, I pretty much get a headache. But what I probably hate more is the person that finds the bug and doesn’t isolate it back to previous releases, saving that fun task for me.

Do you have a mobile app? Not only do you have to think about what versions of the app you support, but also where people can run that app. Android? iOS? Tablet? Phone? The device matrix alone for all the places you could test is overwhelming. Some days I feel like I’m looking at one of these:

Source: OpenSignal 2015
Source: OpenSignal 2015

But we can mitigate it. We can test smart, and we can start using Big Data to pick what is necessary. 95% of your clients in Chrome? You probably don’t need to test in Opera. 50% of your clients in Version X? Maybe don’t focus 80% of your resources on Version Y.

The Client is King

Look, I get it. We have to keep our clients happy. Some are perfectly happy where they are, on something released two years ago. Once you can empathize and see the issue from the client’s perspective, it does make it a little less maddening to have to do all that work. We will never have perfect software, so expect Client Reported Issues. At some point, it is very difficult to be testing the latest greatest feature, then all of a sudden have to work on something that was built five or more years ago. As testers (and developers), we need to come together with Product Management as a team to show them the impact of supporting multiple versions, and how hard it is. What are the impacts to the schedule? What are the risks? What happens if we fix this bug in all versions? As a team, think not only about what you are fixing, but where.

Image Source:
Image Source:

Yeah, I want to smack whoever says that. Like the Product Manager that said we should support something from more than ten years ago. Just, no.

Written by

Ashley Hunsberger


Cross-browser testing