Testing, Not Just a Technicality
User testing. It’s not the most exhilarating step of the software development cycle, but it is just as important as the rest. Unlike technical QA testing, there are many parts of a piece of software that cannot be verified with scripts and automated programs. User experience and design testing is just as vital as making sure that the programs work correctly on the code level. Even without technical errors, a product could fail due to poor execution, confusing navigability, and restrictive or irrelevant functionalities.
So then why is product testing one of the of the most overlooked and underfunded elements of software development?
Why would a development team not spend the time to ensure that their software is working flawlessly as advertised?
Shouldn’t testing just be a given?
Not Enough Cooks in the Kitchen
One of the biggest roadblocks to getting your software properly tested is the lack of available testers. This is a particularly prevalent issue in smaller companies and startups, where the time and manpower is already spread thin. When personnel are scarce, dedicating hours just to verify what you already think you know seems like a tremendous waste of resources. That tester could instead be expanding capability, or customizing solutions for strategic partners, or doing a host of other activities that generate revenue, rather than spending a day or a week or a month making sure that the software does everything you say it does.
Pictured: User testing, metaphorically speaking.
Unfortunately, this well-meaning attempt at efficiency and resource management often ends up costing the company even more time down the road. As users begin to find issues and discrepancies in the software, the dev team has to go back, recreate the problems, and then work on fixing them. Usually the timeline is accelerated because there are actual customers awaiting changes on the other end, and much of the progress that was made by ignoring testing ends up wasted as you scramble to get things back in working order.
But What About the Sale?
Maintaining and utilizing a testing lab is critical not just for supporting existing customers, but also for bolstering your sales. Almost any major customer for a software system is going to insist on a proof of concept or other trial before making a purchase. During that trial, they’ll be seeing your product through eyes entirely different from your development team, or your executive team, or even your sales or marketing team. They have a problem they need to solve, and every experience they have will be seen in the context of that problem. Their assumptions will be different from those you made when you created the software, and it will show almost immediately when they start using your product.
Your best bet in this case is to establish a testing group separate from development and marketing, with a mix of technical and subject matter experts. This approach will allow you to catch most of the user disconnects that pop up during trials and proof of concept presentations. It is also important to remember that as you expand, you must continuously reevaluate and rescale the tests you perform. Testing that only represents a small set of the stakeholders in a project simply cannot predict the reactions of an entire market.
Don’t Test the Users
“So what?” you may be saying to yourself. “What’s wrong with my users being my testers? It means we can focus only on the problems that are relevant to the customer.”
Glad you asked.
In the past, much of the actual feature “testing” was handled during requirements collection and then left as is once implemented. Later, when highly iterative development became a best practice, application improvement became a back-and-forth dialog between users and developers (with some intermediaries). Unfortunately, neither of these strategies are realistic; customers don’t know exactly how to describe what they want or need right off the bat, and forcing users to become testers destroys trust in the product and costs both developer and customer immensely in time and treasure.
Let’s Discuss!
- What experiences have you had with poorly or untested software?
- How can you tell if a product has gone through testing?
- If you’re a developer, what are your biggest challenges with testing software?
- What do you think the most important part of user testing is?
New firms trying to enter markets with well-established or well-funded competitors are especially vulnerable to these dangers. Why would somebody go with your product when a competitor can underbid you with suite of tools forged through years of user use? If your product hasn’t seen a variety of business models and operational circumstances, it can’t be expected to meet the needs of a wide base of customers. When your organization is ready to go from 2 sales to 100 or more, you have to already know your customers’ models and be ready for their expectations. Company-side testing is the only way to do that without costing your company hundreds of sales.
Final Question: How do I do it?
Alright, you’re convinced that testing is worth it, but you have no idea how to go about it, or if your company can even handle devoting the resources to it. This is understandable. Developing a testing team is a tremendous investment of time and resources, both in direct and opportunity costs. It can be a difficult process, and while some companies (such as Deep Core Data, hint hint) have worked out their own strategies for efficient and effective testing, many teams don’t even know where to begin.
In the end, the simplest way to explain a successful testing environment boils down to your team. An effective test team consists of, at its minimum:
- The one who builds the test environments
- The one who plans a test that covers all use cases, no matter how few clients use them
- The one (or ones) who execute the tests and push the buttons
You can double up these roles our spread them out as need be, but make sure that you always go in with a plan, working infrastructure, and the knowledge needed to thoroughly examine every aspect of your product. If you do, you’re much more likely to pass the most important test of all: do I want to buy your product?
I agree with most of this article up until you talk about the “HOW.” You will usually want to get users who are not associated with your company to test your products and get unbiased results. If you need a website’s usability tested, you can use sites like usertesting.com (expensive), trymyui.com (moderate) and userbob.com (inexpensive).
Thanks for the comment, John!
There are certainly merits to getting testing away from in-house teams. That’s why we offer QA and usability testing services (for applications and programs, as opposed to websites) for those smaller companies that are primarily focused on development. It’s good to get the software out of the hands of people who have used it for months or years, and give it a fresh pair of eyes.