The Quality Assurance Process

The Quality Assurance Process can be your best friend. Ignoring it can be your downfall. Is it worth all the trouble? Should you really bother?

I can hear it now: “Oh no! Not more process!! Aaakkkkk!!!” But before you run away screaming, let’s get one thing straight:

Nice and simple, just the way it should be. However, many times during the software development process, “nice and simple” is not the way of things. That is why you have software testing methodology, quality assurance process, and quality assurance procedures in place: a full set of system tools that allow you to adapt quickly and effectively to any situation.

How does this work?
Let’s take a look at the overall setup.

You should have defined Milestones that created a framework for Software Development Lifecycle. Once implemented, you and your QA team should know:
• When you will receive a software delivery
• What assets and functionality each delivery will contain
• What your testing will entail once you receive the software

With all of that in place, you can more clearly define your Quality Assurance Process. The Quality Assurance Process that you implement should meet the specific needs of your team and the type of software you are dealing with. And remember that although there are many constants, there is no one-size-fit-all when it comes to creating effective Quality Assurance Process.

The specifics of how you will handle each part of your Quality Assurance Process are defined in your procedures. What must happen is your process – the HOW of making it happen; the step-by-step what-to-do, that is procedure.

So where do you start?
How about at the beginning…

Review Documentation & Meet with Developers
This is a critical early part of your quality assurance process. Whether these documents are electronic, on paper, or on a whiteboard as ideas, your QA team needs to be involved this early in the process. Early inclusion of QA most often prevents the creation of several bugs – there is no less expensive way to fix a bug than to prevent its creation.

If you are working from detailed documentation, review it thoroughly for any and all items that may cause a problem when building tests and while testing. Make note of absolutely everything – it is less cheaper to clarify a misunderstanding than it is to rework code

In this phase you should also interview the designers and developers about what they see as the most problematic parts of the development. If you can understand what they perceive to be potentially difficult areas, you can take steps now to mitigate their impact on your testing later.

Although this part of the quality assurance process is more experience than procedure driven, as you practice the minimal procedures you will develop greater expertise. When you have completed this phase of your process, you should have a very clear understanding of what problems you will face going forward. You should have a very good idea of how to best spend your testing hours.

Procedures:
• Review all documentation
o Note all ambiguities, conflicts, contradictions, etc.
o Note anything that lacks an adequate description
o Note functional dead ends
o Note non-uniform UI features
• Meet with Developers
o Ask for technical complexity assessment
o Request opinion of high risk functionality and implementation
o Gain understanding on any issues that remain unclear

Submit Review Feedback
This is where you give the designers and developers your expert feedback. Having dissected all of the project documentation and taken careful notes, you are ready. In this critical phase of your process, you will deliver a detailed report to the designers and developers, respectively, which highlights all of the discrepancies you have found.

The more issues that you can get resolved in this phase, the smoother your project will go. The action you take in this phase of the process results in many bugs getting fixed before they are ever coded. Face it, there is just no cheaper way to fix a bug than to make sure it never exists in the codebase.

Your work in this phase combined with your work in the previous phase can have greatly beneficial effect on the project. This is work may seem trivial, but don’t discount how very impactful these proactive measures can be. Although not sexy, QA is most efficient and cost effective when used preventatively. This is a prime example of just how to employ it!

Procedures:
• Deliver report to entire project team using Document Review Feedback Template
o List all items needing clarification
o Highlight all items requiring redesign due to conflict or contradiction

Side Note #1:
In the most efficiently and effectively managed projects that properly employ the quality assurance process, these last two phases (Document Review and Feedback) are repeated until the project documentation is in a reasonably cohesive form and all project members have a clear vision of what is being create. This should not take more than three cycles, and if performed with due diligence the first time may not even take two.

Side Note #2:
In some development teams there is little to no formal documentation. In these cases it is still imperative to include QA if you want a smoother running, less expensive project. A true QA Professional can tell the designers and developers what common pitfalls they are walking into.

A QA Professional worth their salt will be able to bring a unique perspective that will help the team think through their design and implementation more thoroughly. Their unique perspective allows them to identify potential bugs while still in the design phase. Their experience enables the team to get the project planned and underway more quickly, while at the same time setting up the project for greater success.

Build Test Suites
In this phase of the quality assurance process you use the project documentation, your notes from the documentation reviews, and all information gleaned from meeting with the developers. Take all of that information and start building your test suites. Whether your tests will be automated, manual, or a combination of both, now is the time to start creating the specific test cases that you will execute when the software is delivered.

There are many, many books on the process of building tests –I won’t attempt to summarize them here. The key to this phase if to get started documenting the steps you will take to ensure you are testing all of the code. Break each piece of functionality into its simplest components and create test cases that will prove or disprove each component’s existence.

Once you have done that, be sure that you have included test cases that will test the overall functionality. That is to say; make sure that ALL the components work together holistically.

Procedures:
• Begin creation of tests using QA Test Suite Template
• Save all work and maintain readiness for its review
• Upon completion of test creation, save clean backup copies to secure location

Side Note:
Some organizations like to have the entire project team – or sometimes just parts of it – review test suites as they are created. Some organizations bypass this entirely. Either can work if the rest of the overall process and methodology is adhered to with due diligence. It just depends on your unique version of the quality assurance process.

I am not here to say one way works better than another. What I will say is that the higher the complexity of the software you are developing, then I would recommend having at least a brief sit down with the developers to get their feedback on your approach. By the same token, if the software is not considered all that complex, but you don’t feel you have a firm grasp of it, sit down with the developers to their feedback on the tests you are planning.

Take Receipt of the Software
This is an official hand-off and should be treated as such. How formal or casual is up to you, but there needs to be an official communication from the QA team when they take receipt of a software delivery. This ensures that the hand-off was successful for the deliverer (them) as well as the recipient (you). Send an email, a memo, or whatever other written, TRACEABLE form your company uses to alert the entire development team that QA has the ball (the software).

No mater what other wrinkles you do or do not employ of the quality assurance process, ALWAYS take this step. Trust me, it's worth the minimal effort.

Procedures:
• Verify delivery of build
• Verify delivery of build notes
• Verify all necessary changes have been made to bugbase
• Report QA’s receipt of all of the above to the entire project team using Build Receipt template

Perform the BAT
What you need to know next is whether the delivery will be testable or not. So the next step in your quality assurance process should be a way to evaluate that. This is commonly known as a Basic Acceptance Test or Smoke Test or Sanity Test. Follow your testing procedures to determine whether you will go forward in testing the software or reject the build.

Why would you reject a build as not testable? Because in the methodology that you set up with the project team, each deliverable (and this is one) is supposed to meet pre-defined criteria. If it does not meet the “Pass” criteria, you have the option of saying “no”. “No, this does not meet the criteria that we previously agreed upon.” You do not have to spend the company’s money testing a build that does not meet the preset build criteria.

Procedures:
• Execute BAT
• Evaluate BAT results
• Determine QA response based on BAT results
• Report QA’s test results, acceptance/rejection of build, conclusions, and planned actions using BAT Result Template

Execute Planned Tests
Every build delivery that you receive should have a battery of tests planned for it. You defined these in your Test Approach when setting up your QA Methodology. In this step of your process, you use the detailed Test Suites that you have developed earlier in the quality assurance process that contain the steps for each specific Test Case.

That’s really it for this phase of the quality assurance process…Run your tests.

Procedures:
• Execute tests
• Log test results
• Monitor progress and adjust testing priorities as necessary

Deliver Test Results
Once you have completed all of the tests planned for the current build, assemble the results and prepare them for delivery to the project team. Compile your results in an easy-to-understand format (since not everyone on your project team reads and speaks QA).

When you send your results, you must include information to help the rest of your project team understand what you are delivering. State what you did – what tests did you perform. Explicitly refer to the attached (or accompanying) test results. EXPLAIN what the test results mean (see more below). Detail current QA status.

When you deliver your test results, it is imperative that you explain just what the results you are delivering mean. What do they tell you? Don’t assume that the rest of the project team will understand a bunch of numbers that you throw at them. Don’t assume that they will come to the same conclusions that you did about the testing that you just performed.

Tell them what the testing showed you. Make explicitly clear what the tests that Passed tell you and what the tests that Failed mean. You have told them what testing you did, you are delivering the results – now you must clearly quantify those test results. Executing this part of the quality assurance process well will set you apart from the crowd.

Anyone can push a button and log a result. One of the things that separates a QA Professional is their ability to make an intelligent analysis of that result. It is unfortunate that this little detail is lost on so many who claim to be Software Quality Assurance Testers. Any marketing intern can push a button and log a result (no offense to my marketing brethren). Any automaton can compile a bunch of results into a sheet full of numbers and deliver them. But the QA Professional makes sense of those numbers in a way that is useful for the project and the project team.

Procedures:
• Compile results of current round of testing into standard QA End of Build Report Template
• Translate test results for the project team
• Deliver test results with accompanying translation and project status update

Update QA Status
This is often thought of as an optional phase of the quality assurance process, but should not be overlooked. Once you have finished testing the build and have delivered your report, you should publicly position your QA team. You need to communicate somewhere publicly for the whole project team to see. State that QA has completed testing the build and is preparing for the next.

Whether you state this explicitly and in an obvious location in your End of Build Report email, update a status on a company Wiki, Quality Assurance project dashboard, or other – be sure to do it. Your goal is to make it clear that you are finished with the most recent build.

Procedures:
• Update QA status in a public forum

File Results
Capture all of the test results from your most recent round of testing. File them in a central location where they will be available for future analysis. You can use these historics for reference when compiling your End of Project report, holding your project Post Mortem, and for updating your metrics that you will use when estimating future tests.

What you measure matters – this is a key point in the world of Quality Assurance and following the quality assurance process makes this possible. If you want to create the opportunity for improvement, you must first have a baseline starting point. Measuring your results gives you this baseline. Continued measuring gives you the data you need to create a plan for improvement.

File all of the testing results you can capture without it becoming burdensome. Start small. Capture a few numbers first. As you get used to this and it becomes easier for you, expand and add more data, just remember to always file it away somewhere safe so that you will have it for reference whenever you need it.

Procedures:
• Capture all relevant test data
• Compile captured data in form suitable for long-term analysis
• Update all relevant metrics using latest compiled, captured data
• Archive data

Prepare for Next Build
You have worked your quality assurance process as you had planned. You’ve done everything that you were supposed to; you set up for the project proactively, prepared your testing, executed your tests, compiled, translated, and delivered your test results. You let the project team know that you completed work on the build, archived your relevant data, and are ready to take a breath.

Ah, if only that were the way of things. If you live in the real world, chances are that you now have enough time to make sure that you have clean, updated test suites that are ready for another round of abuse. Get your team ready to go and bring on the next build!!!

Procedures:
• Ensure availability of updated, ready-to-use test suites
• Get all QA team members on same page
• Prepare team members for next round of testing







Return from Quality Assurance Process to Successful Quality Assurance Home