What Quality Assurance Software Do You Need?

To most effectively demonstrate your true value, you need effective quality assurance software. Since you, as a QA Professional, work so hard to continually bring value to your projects, make sure you have a tool that enables you to highlight that value.

If you are using a cheap (not-necessarily inexpensive) tool, you are wasting your time, your company’s money, and the time of everyone else that has to try to use the software you champion. As long as you are going to be looked to as an expert in the field of quality assurance (that is one of your goals, right) you might as well use and supply tools that add value, not remove it.

One key way to do that is with solid quality assurance software. How do you know which quality assurance software to use? Which tool is right for you and your projects?

Each individual situation is a little different and each set of users has their own collective unique needs, so picking which quality assurance software can be tedious. But if you understand the principles, if you have a clear picture of what end result you need, then you will know what to look for and this entire process becomes much simpler.

What is Quality Assurance Software?

Quality assurance software is a tool that you use to track various aspects of your project. In its most basic form, quality assurance software is a defect tracking system commonly referred to as a bugbase. All imperfections in the program and deviations from spec are logged in the bugbase.

The bug tracking aspect is the most critical of all features quality assurance software provides, but it is not the only feature. Many of them provide a whole host of bells and whistles. The more feature-rich tools include entire project management features, scheduling, single-button report creation, use case templates, automated testing interfaces, etc.

I have used no fewer than twenty differ quality assurance software packages over the last ten years. Each has their own benefits and drawbacks. Most are just fine for the specific project, or project type, they were developed to handle but there ends the fun.

Very few quality assurance software packages really contain enough options for a real quality assurance team. Of those that do, most are thought to be too expensive by many executives and middle managers. Intimidated by the large upfront cost, many companies opt for a tool that really does not meet their needs.

This is what I am trying to help you avoid. A true quality assurance software tool is worth what it costs. It will return what you paid for it in accuracy, efficiency, and organizational support. Look for a tool that meets your needs.

What Are Your Needs?

To choose a tool that will fit you and your project, you need to know a few things:
• Who will be using the tool every day?
Which department will be using this tool each day? How many people and what level of use will your quality assurance software be subjected to? Make sure you take into account how many licenses (if not free software) you will need and of what type (single-user, floating, etc.). Also make sure you know that you have the hardware and bandwidth capable of handling the traffic load that you anticipate

• Who will be using the tool only occasionally?
Know how many licenses you will need for VIPs. See if you can secure licenses that can be shared for short periods of time or get an option for inexpensive temporary licenses.

Also note if your VIPs will need to see a quality assurance software solution that has bells and whistles. Sadly, sometimes you have to get software that looks pretty so that the brass will let you do your job properly.

• How long do you plan for this quality assurance software to be your solution?
Will this be a temporary stop-gap solution, or are you looking for a long-term answer? Be wary of the promise that this acquisition will only be temporary. If you get a tool and it is working (even not very efficiently) you may have extra work ahead of you convincing those that control the purse strings that you need a better quality assurance software solution.

Even if you plan for this tool to be temporary, do your work to make sure that it meets your needs. Poor quality assurance software offerings will waste your time and money. Be very cognizant of the amount of effort that will be involved to setup and maintain this tool. And be very careful to select a tool that will assist and not hinder your project.

• What kind of reporting will you need it to provide?
Every project needs reports of some kind generated. Depending on your organization or project team, those reports may be very simple or they may require some serious bugbase queries. Do you know what sort of reports you will need to generate?

If you will need to generate in-depth reports often, you should consider finding a quality assurance software solution that has a robust reporting module. This family of software tools varies widely when it comes to reporting capability and complexity. You will want a solution that is right for you.

How long will it take for you to set up your initial queries? Can you save them? How easy is it to add queries as you progress through your project? These are critical questions to have answers to before you decide on a tool.

As you weigh your options, consider again who will be using the tool and what reporting capabilities they will need. Then consider their technical skills; will you have to set up all of the queries and reports for the entire project team because it is a complicated process?

Selecting a quality assurance software solution that has a full-featured, easy-to-use query and reporting module is often worth the price. Going cheap here means that you will be paying for making the feature work with your time. How much do you cost per hour and how many hours are you going to spend creating reports that have value?

• Do you have in-house expertise to set it up?
Another area of great variation in this family of tools is the area of set up. Will it be plug-and-play (as much as anything ever is)? Or will it take you a month to get installed and configured?

Will you have to do all of the setup yourself? Do you have an IT department with the time to help you set it up? Do you have anyone on your QA Team that has the skills and time to set this all up (not an uncommon occurrence)?

How will you set it up? Where will you set it up? Does it come with any free tech support or if you get stuck are you really on your own.

This is often overlooked until after a tool is acquired. At that point you are at the mercy of anyone dangling a carrot of “help”. Don’t fall into that trap. Know what you are getting into before you decide on a solution. You will be thankful you did.

• Do you need a client solution or is web-only going to work for your team?
Web-based interfaces are usually less feature-rich than a simple client. If you have to work through the web UI then you should also count on a slower experience. Web pages having to reload, refresh, etc. will cost your team time throughout their day. If speed is not a concern, then make sure that the web UI features have all that you need.

I have never experienced a web-based interface that had all of the features as the client. The features are not as well laid out for ease-of-use, each click requires several seconds of loading time, and the interfaces are without fail not as easy to read quickly. I have even seen non-QA project team members get so confused with a poor GUI that they refused to attempt to use it anymore!

Find out if the solutions you are considering have both a web and a client option. Then look at each to evaluate their usability. Yes, this seems like a lot of busy work but once you get a superior tool that meets your needs, you will save hundreds of thousands of dollars (or more) over the years. **see “How Much Should You Pay?” below**

• Are there any test suites or automated tests that will need to connect to it?
Many quality assurance tools come pre-built as a package. You can purchase a single piece, or many connecting pieces, or the whole enchilada. These quality assurance software solutions are built around your bugbase, then they add test tracking capabilities, test automation features, etc.

When launching your solution, will you need anything other than a bugbase? If so, it will probably be least expensive in the long run to purchase a set of tools that were designed and built to work together. This is a far less expensive choice than the alternative – trying to duct tape together incompatible solutions that don’t communicate smoothly.

• What search capabilities do you require?
This is a BIG one! Overlooked far too often when acquiring a quality assurance tool, the lack of an integrated, fully-capable search module prevents anything that calls itself quality assurance software from being a viable solution. Any potential solution that doesn’t have necessary search abilities will handicap your daily work and become an unacceptable nightmare when you realize that you are unable to measure important data – because you can’t gather it!

If your only option for capturing data is doing so by hand as you go or by sifting through each bug, Open and Closed, one at a time – you will “rue the day…” (as they say). As a Quality Assurance Professional, you should know that you need the ability to run any search you choose at any time. Setting up these searches should be quick and painless if not automatic.

In addition to you, your project team members will want to perform their own searches. You undoubtedly will be asked for data, at some point, that you did not anticipate (probably from a VIP who has no idea that the data they are asking for has little or no value – but that’s a different battle).

If the search module and/or search capabilities of the quality assurance software solution you select are not sufficient, then you will at least be performing extra work (that could otherwise be avoided) and at worst will not be able to capture and analyze critical data that could make you product better, save your company money, or show you efficiencies you could implement.

When shopping for a quality assurance software solution, above all else, make sure that it has the searching and reporting capabilities that you need. It should have (or allow) all of the fields that you want to use for bug tracking, and should be simple enough that everyone on your project team can use it. Any other tool you choose will create more work for you day after day after day…project after project. Choose wisely.

How Much Should You Pay?

In the rush to save a few dollars when selecting a tool, companies don’t consider the long-term cost of their decision. Quality assurance software that is not a good match for the projects it will have to handle will cost every project that is forced to use it.

Although a preferred option than “no-tool-at-all” or tracking bugs and features through a spreadsheet alone, a poor quality assurance software choice will exact its toll day after day after day. If the search feature is so poor that it forces users to sort through each issue one at a time to gather any data, or if it is not simple to use it will waste your time.

If your software solution makes users work to get data from it, or understand its UI, its data output, or its input features, it will cost you. If the tool you select is so slow that users have to wait after entering any data or command, it will cost you. If you have to reformat, translate, or otherwise modify in any major way the data analysis, reports, etc. that it produces, it will waste your time.

How much time could quality assurance software really waste? And how much could that really cost you? Let’s look at one simple scenario:

Assumptions:
• 20 project members will use your quality assurance software solution
• Project length is 3 months (20 working days per month) = 60 project days
• Average employee/consultant cost = $50 per hour

You or your company decides to select “cheap” quality assurance software. This results in each project member wasting an average of 10 minutes each day due to the complexity, difficulty of use, or otherwise need for extra effort on the part of the user.

Cost:
(10 min/day x 20 users) x 60 days = 12,000 minutes total
12,000 divided by 60 (60 minutes per hour) = 200 hours total
200 hours x $50/hr = $10,000

What if there was more than one project? Could you assume a $10,000 loss for each 3 month period of 20 people? In truth you could probably assume a far higher total than that. And that loss doesn’t even begin to count the ways in which your quality assurance software selection is adversely affecting your ability to improve. You will be working as hard as you can to simply maintain quality – you won’t have any chance to improve it.

If you can’t see what is going on in your projects; if you don’t have insight into where you are losing money or could add efficiencies; if you can’t see which parts of your project are creating the most issues; then you don’t have the opportunity to ever raise the quality of your offering. A poor choice of quality assurance software will cost you money every day and beyond the easily quantifiable cost, it will preclude you from raising the quality bar for you team.

What Software Solution is Best for You?

If you know what your hard cost will be if you make a poor choice, that should give you a ballpark figure for how much you can spend upfront and recoup by simply using your solution. Using the simple numbers from the scenario above you can see that even just a few projects will provide you the cost savings you will need to justify a purchase that might otherwise seem overly expensive.

I am not saying that you should buy the most expensive, bells and whistles laden, off-the-shelf solution you can find. There are several free and open source quality assurance software solutions that may fit your needs. Note I said may…there are MANY that will not.

If you want to be taken seriously as a QA Manager that a team will follow, you better make sure that you have done the background work to make the right choice. If you are going to take your career as a Quality Assurance Professional seriously, learn which tools will work for you and which won’t. Learn it now, you will be glad you did later.

Choose a tool that will enhance the processes you have in place. Adjust your procedures to most effectively account for your solution’s features and shortcomings (just on the off chance it has any). Make sure that your setup supports your testers in their activities and enhances their efficacy. Do not shackle your team with an albatross.

You must find a solution that has the options you need, is customizable to your projects, and is simple enough for all of your team members to use (see the section 7. Bugbase Maintenance of our QA Lead page for additional help). This may an off-the shelf solution or it may be free or shareware. Take all of the factors you researched into consideration before you make a decision. Do your homework and you will find a solution that is right for you.

You must ask the right questions if you want the right answers