Automated Software Testing

They shoot testers, don’t they?

What Is It?
Automated Software Testing is testing that is executed using a tool (either off-the-shelf or created in-house) to exercise code. These tests are set up by a QA tester or QA engineer using the tool so that a battery of tests can be run by pressing a single button or entering a single command.

The tool may be complex or it may be a simple point-and-click interface that records the actions the tester takes. Once the test steps and actions to be performed are defined, they are recorded so that they can be executed any time in the future.

That, in a nutshell is it. Automated tests are scripts or “instructions” that run a suite of tests without any monitoring necessary until they have completed.

That is where automated software testing provides value – in running independently. While tests are running automatically, human testers can go about other QA matters; manual testing, reporting, automated test development, etc. This allows a QA team to leverage their time for more significant impact.


The Benefits of Automated Software Testing
Once your tests have been created and adjusted so that they can provide accurate results, then Automated Software Testing will begin to pay off. Once in place, it significantly reduces the amount of time taken to perform repetitive, otherwise time-consuming testing. Thousand of tests can be run in a matter of hours and minutes, not days and weeks.

This time savings is also felt in monetary savings. All of the hours of testing that a computer can supplement or replace, hours that it would take a human to run the same tests, provide a great deal of savings to the project budget. Just think of how many commands, simulated button presses, and mouse clicks, how much data entry, variable configuring for compatibility, and automatic reporting can occur in while the QA team is in a meeting – or how many tests can be run overnight while the team is sleeping. The addition of accurate Automated Software Testing will enhance the efficacy and efficiency of any professional Quality Assurance Team.

So everything should be automated, right..?

Automate or Bust!

A fool with a tool is still a fool.
~ Old Maxim

No computer can quantify the end-to-end, holistic experience necessary to properly value human targeted software. Automated software testing can verify the functionality, the limits, and the options, but it cannot replace human judgment.

Cost Savings Takes Time
The time and financial cost savings to be gained from automated software testing take time to realize. Automated software tests don’t simply appear out of thin air and begin running flawlessly. No, they require expertise, implementation, adjustment, and maintenance.

First you must select a tool. What automation software are you going to use? Will you develop a proprietary automation engine in-house? Or will you purchase one off the shelf?

If you develop your automated software testing tool in-house, it costs the time of your developers. This is time they could be spending creating software that the company could market for profit. And the developers will need to be paid for that time, and that is just immediate hourly (or salaried) cost.

If you choose an off-the-shelf tool, then all you have to pay is the cost to buy it. Easy. Did I mention that a single tool (not an entire suite of tools) can cost thousands of dollars? Depending on your needs you might be able to get away spending as little as a few hundred dollars, but if you need a tool with any teeth, you are looking at a hefty sum. A single tool can cost tens of thousands of dollars – so choose wisely.

Either choice requires that you end up with a tool that will meet your testing needs. That requires expertise. Are you the person who will make the decision about which tool is the best fit? If it’s not you, do you have someone on your team that can explain which tool would be best? Or will you have to find and pay an outside consultant to choose a tool and set it up for you. This will cost!

Once you choose a tool, you must implement it. That means creating your first batch of test suites. Is your automated software testing tool one that you know how to use? Does someone on your team know how to create scripts with it? Or again, do you have to go outside the company and pay someone else to do this for you? Once again, this is going to cost.

Even if you are able to get up and running with all of the talent and expertise you have in-house, to set up a testing pass requires time; time to set up your test scripts and make sure that they work. This takes more than a couple of minutes.

Once you have defined and recorded the steps, you must run your automated test scripts and fix any issues that will produce a falsely negative result. What I mean by this is that you must ensure that all of the results your tests produce are reliable. Each failure your test script reports should be an actual failure – not a false alarm. This is why you must take the time to create and then adjust your test scripts.

After you have set up your automated test scripts, they must be maintained to keep up with the changes that are made in the product you are testing. If, for instance when testing a web site, your scripts begin reporting that a link is not working, be sure that the link itself is bad. If the placement of the link on the page has changed, but your test script has not been updated to take this change into account, then you will see a failure. But this failure will be a false negative. If you report this false negative, you will appear to not know what you are doing.

Can you see how it will take some time to reap the benefits of automating some of your testing?
• Choose or develop a tool
• Implement the tool
• Adjust the tool
• Maintain the tool

Skipping any one of these steps and you severely decrease the return on your automation investment. Automated software testing can be a great help, but it can also be more trouble than it’s worth.


Should You Automate?
This is something that you will have to evaluate on your own. As you can see, automated software testing can save a great deal of tester time, allow testing when no one is around, and establish stable replication of any test you can implement. These are all great improvements to any project.

But automated software testing can also incur heavy cost to get off the ground. If you don’t create solid processes around maintaining your tests, it can also cost a great deal to keep your automated software testing up and running.

And remember, even with much of your testing automated, there is no substitute for the judgment of a qualified QA Professional. Make sure you consider your need (or lack thereof) for human analysis during your project and how you will incorporate it in your project

So, should you automate or not? As I said before, this is a choice that only you can accurately measure. Do the strengths of implementing automated software testing outweigh the cost of implementation and maintenance? Will your project last long enough or have enough similar revisions to realize the cost savings that the automation intended to confer?

As you consider whether to automate or how much to automate, keep the following questions in mind:
• How long will the project last?
• Will your planned automation be applicable to any future projects?
• Do you (or your team) have the necessary expertise?
• What sort of tool will be most effective? And what is the price range of that tool?
• Does automation fit with your testing methodologies ?
• What is your project budget? Can you spread the cost across multiple projects?

Those questions should be enough to get you started thinking along the correct lines.

Remember that whatever you implement, it should fit in with your overall methodology and processes. Make sure you take the time to implement proper procedures so that your automation is a strength and not a weakness.