Quality Assurance Procedure

The real key to Quality Assurance Procedure is taking the time to do the right things first. Although many of the steps may appear on the surface to be extra busy work (to the untrained), taking the time to do the right things first will save you and your team huge amounts of time, stress, and worry later.

Just as with any other system, a poorly implemented Quality Assurance Procedure is going to waste time. Don’t fall into that trap. Don’t get stupid. This is a simple system. Each step is designed to make your work life easier down the road (not too far down the road). If any step in the Quality Assurance Procedure that I have listed doesn’t apply for you, then skip it.

The idea here is to create more time for yourself and for your team; more time and less stress. To do that, you must evaluate and asses as you go (you do test…don’t you?). Determine which steps, if any, are not effective for you and remove them from your system. Don’t perform steps just so that you look busy.

You may need to try all of the Quality Assurance Procedure actions listed below in order to determine what works for you and what doesn’t. That’s great! Do that…but do it only for as long as it takes to pare down to the most efficient and effective system for you. Your development process may not need all of these steps…or…it may need more. Once you have a handle on the overall process and what makes it go, and you are familiar with Quality Assurance Procedure, you should be able to adjust and tweak as necessary. Add and subtract to make the best system for you.

No matter if you are an entry-level QA Tester, a veteran Quality Assurance Lead, or a new and improved SQA Manager, you must understand how the quality assurance procedures that you implement and follow are the engine that drive your processes and methodology.

Here are the Procedures that make the real-world example on the QA Process page come to life…

1. Review Documentation & Meet with Developers

When reviewing documentation, your goal should be to find anything that will slow down your project. There are several ways in which a project can be hindered by bad documentation and your job is to flush these out and get them resolved. To do that, you will review all of the project documentation; marketing specs, production specs, technical specs, look-and-feel design specs, schedules, etc.

When you review the documentation, make note of all ambiguities. The specs should be clear. If they are not, you can easily plan to test something different than what is being developed. The developer might create something different that what was desired. The docs should be clear, ensure that they are. Any statements that are unclear, an items that are not well defined; these are fair game for you to get more details about. This is as valuable a Quality Assurance Procedure as you will employ. Think preventatively!

Look for conflicts and contradictions. In the process of creating project documentation if is very common for the writer(s) to contradict themselves. This can happen because 2 or more people are writing the docs and they each interpret an item in their own way. Or because design changes occur in the middle of writing the docs – thus any previous reference to the changed area is at risk.

Look for “box canyons”. This is a functionality that, although properly documented, will have no exit. The user will follow their defined path and then arrive at a functional dead end. Make sure your customers can always navigate your product with ease.

Find any User Interface design that will be unique to your software. These should always be questioned. Although innovation is wonderful, the consuming public expects standardization in their interfaces. If your software has created a “better” way to do it, make sure there is a good reason for it.

Meet with Developers
After (or in the middle of) review the project documentation, you should meet with the developers to get any questions answered. Get the developer’s perspective on the project and have them explain any areas that you have questions about.

Ask, from a development standpoint, what the biggest concerns are regarding implementation. What are the potential problem areas – and what testing can you do to help the developer gain insight or confidence? Ask what the high-risk areas are. Discover how to prepare for the issues that the developers think will arise.

Side Note:
It is not at all uncommon for developers to think that they have all the potential hotspots figured out. That can complicate this step of your Quality Assurance Procedure. They may tell you that it won’t be any big deal.

They may be lying to themselves on purpose, they may not want to share their expertise with you, or they may be in denial. Either way, remember that it is your job to uncover these areas of concern so that you and your team can be ready for them…End of speech.

If, after reviewing the documentation and chatting with the developer(s) you are still unclear on anything…ASK! You are having this conversation with the developers to gain insight and clarity. So, anything that is not crystal clear yet…ASK!

2. Submit Review Feedback

As you fill in this report, you list all items that still need clarification. Just because you identified that more information was needed does not mean that it was readily available. People may still be working to gather the necessary information (if they had it in the first place, you probably wouldn’t have flagged it as an issue, right?).

You will also make a list of all remaining items that require a redesign or redefinition due to some conflict that you identified. You explain in the report which areas of the project remain at risk due to ongoing contradictions in documentation.

No, this really isn’t the glamorous work. It also shouldn’t take you weeks to perform – just a few days for a document-heavy project. But the return you will get from this time that you invest is invaluable. Make this a key factor in determining which steps will work for you as you develop your own Quality Assurance Procedure.
You will save yourself and your team a large amount of stress and headache by getting the docs clarified before you are in the middle of the project. Once everyone is running around too busy to answer each other’s questions, it is too late. These potential bugs are cheaper to fix while on paper. And your testing will be more effective if you are able to concentrate on the actual software and not all the confusion that surrounds creating it.

3. Build Test Suites

Using the information in the documentation that you reviewed as well as more in-depth insight from the developers, designers, etc., you begin creating your test suite. Create all the cases that you will need to verify the software that is being developed.

Once you have created a functional test suite…Save it.

Save All Work and Maintain Readiness for Review
Now that you have written the test suite, it is critical that you save it somewhere safe. This seems like it should be no big deal, but you would be amazed at how often these documents are created and saved to a personal hard drive, or a thumb drive, or a CD…only to have the only copy disappear. Hard drives crash, thumb drives and CDs can be lost.

Save your test suite(s) to a repository on your server (in the QA domain) that gets backed up regularly. Ignore this warning at your own peril. You will need access to them for edits and review. And you will want to save each revision uniquely so that you have a reference for future projects (as well as an enhanced documentation trail). This is detail that is critical to using effective Quality Assurance Procedure – save what is useful, always.

Once saved, key members of your project team should review your test suites. This will help the developers understand your approach and show what specific actions you will take to test their code. Ask for their input. If they have any suggestions about how to test something more accurately, make note of it. Help your project team understand what you and your team will be doing.

Upon Completion of Save Clean Backups to Secure Location
After your project team reviews your test suites, update them with the appropriate feedback and save the new version. Again, make sure you test suites are saved to a safe and secure location. Now is not the time to start over from the beginning.

Save a “clean” (no test results) version of the test suites that you can use for each round of testing. This simple step will save you having to delete results from tests that you performed in the previous round.

4. Take Receipt of the Software

The value of these simple steps is not readily apparent. They may seem like a nuisance and maybe you would rather just move on…but don’t. Find the quickest, simplest way to do these steps, but do them. This is where your Quality Assurance Procedure will provide its great value.

These are the right steps to take first. Solidify a quick method and enjoy the confidence that you are on track on all on the same page. Ignore these steps at your own peril. If you or your teams spend hours or days testing the wrong thing or testing the right thing in the wrong way, then it will be your fault that you are behind schedule.

Verify Delivery of Build
Your test suites are ready and so is your team. Once a build is submitted to your team, do a quick check and make sure it is what you are supposed to have.

Does it match what the schedule says you should have? Does it have an appropriate build number? Is it in the proper format for your team to use?

This should only take a few moments, but now is the time to spend those moments. If you release a build to your team that is the wrong thing, you can create a lot of confusion…so don’t. Take a couple of minutes to see that you have what you are supposed to have. Remember, effective Quality Assurance Procedure should ease your work life by being simple and preventative.

Verify Delivery of Build Release Notes
Again, a very quick check. Every build should be accompanied by Build Release Notes (and shame on any developer who delivers otherwise…shame).

Do the Build Release Notes list the items that you expect?
• Build number / Version number (make sure it matches)
• Release Date
• New Features
• Known issues
• Fixes (and associated bug numbers)
• Etc.

Are you able to determine that these Build Release Notes match the build you just received?

Verify All Necessary Updates to Bugbase
Check to see that the fixes listed in the Build Release Notes are reflected in your bugbase. If there were fixed made, then there should be bugs assigned to you (and/or your team). Check now to make sure this is the case.

If this hasn’t happened yet, get on whoever is assigned that task so that they will begin updating now. Insist that interaction with your QA team follow effective Quality Assurance Procedure - everyone will benefit. That way your team can begin, and by the time they need to reference the bugbase, the changes should have been made.

This means that when you receive a build, the Build Release Notes should say more than “we fixed some stuff about logging in.” They should detail what was changed, what bug numbers were affected, and those bugs should be Resolved in the bugbase by someone other than QA.

Report QA’s Receipt of All the Above with Build Receipt Template
Once you have verified that you have the right build, the right notes, and that the bugbase is ready…it’s time to send out your “QA has the ball” report.

Being the organized professional that you are and knowing that employing proper Quality Assurance Procedure is your most effective means, you whip out your handy-dandy Build Receipt Template and fill in the necessary information. What this basically says is that:
• QA has the build (include build number)
• QA has the accompanying notes
• QA acknowledges updates to the bugbase
• QA will begin the BAT
• QA will update soonest

There you have it! All that juicy information – and contained in no more than 5 lines…done! Compile the report, save yourself a copy (in a safe and secure location), and send it out to the project team.

5. Perform the BAT

No matter how you run it, this test suite should be quick, short, and to the point. Key pre-defined areas should be targeted for testing, with specific expectations of what is “acceptable”. Follow the steps you already defined. Run the test and log the results

Evaluate BAT Results
Once you have finished running your Basic Acceptance Test, look at your results to determine your next course of action. Your results should give you enough information to bless the build and move forward with testing or reject the “weak sauce” that was delivered to you.

Analyze your test results. You should have clear criteria already defined to assist you in your decision to accept or reject the build. Did enough areas Pass enough of your tests to accept the build? Or were there too many Failures? Is there enough functionality to expend your team’s time on?

Determine QA Response
Based on your evaluation, make a decision whether you will accept or reject the build. Your decision will need to be accompanied by the hard data that you assembled when testing. Your response should include this data, delivered in easy-to-understand language.

Report QA’s Results, Response to Build, Conclusions, and Planned Actions Using BAT Result Template
Since you are remaining true to your Quality Assurance Procedure, you go grab the next template in your arsenal. Here you log the specific Pass/Fail results from your BAT along with all of your supporting information.

Your supporting information should be a clear translation (for your readers) of what the impact is of each major failure. You will insert it as needed to clarify any risks or rejections that you state.

If you are rejecting the build outright, then point out the failures that lead you to this decision and explain why they are a reason for rejection. If you can’t test anything because the build is broken, state that (in a nice, professional manner…of course). Communicate that QA is rejecting the build, list your reasons and reference the failures, and clearly state what QA is going to do next (await new build, continue to refine test suites, go to a movie, go fishing…be clear)

If the build is in wonderful shape and there were no major failures that point to elevated risk, then be happy and share the news. Tell the project team that QA has blessed the build and will proceed with planned testing (they know what tests you will be running because they reviewed them…right?).

If your BAT results fall somewhere in between these two rather straightforward scenarios, then you will just have to use your professional expertise. You may find yourself in a scenario where a build is testable “enough”. It might be a build that you would reject otherwise, but due to outside factors (like a compressed schedule and need to get to market) if you don’t start testing, the project is over.

When this happens, if you decide to go forward with testing, make sure you communicate the high-risk and waste of time the build is creating. Then include (very diplomatically, of course) that your QA team is doing their best to compensate for the missed deliverable(s). This is not only solid Quality Assurance Procedure, but it also shows how valuable your team is.

So, the build that was delivered is not what it was promised to be, but you will be able to salvage enough useful testing out of it to help the project creep forward. Make sure you are very clear and direct in communicating that your team is taking this action to help facilitate keeping the project moving forward.

It’s fair to report that: “The build is clunky, funky, and smells like a monkey, but our QA team is going to do our damndest to keep this project going. Even though someone else is once again dropping the ball, QA will do our best to save you.” (Ok, so that’s not exactly textbook Quality Assurance Procedure, but I think you get the idea – feel free to communicate in a more diplomatic voice if you choose)

Your report should show all of the failures and explain the impact of each, just as you would if you were rejecting the build. Then you will explain that QA is going to proceed with testing that you explicitly call out in the report, state that this is due to the even higher risk of not testing, and clarify what action QA expects from other project team members in order to support this decision.

What I mean by that is that it is well within your prerogative to say that QA is going forward. But for the project to not incur even higher risk of failure, QA expects an additional, functional build to be delivered ASAP.

Since the Quality Assurance Procedure you have implemented states that you should use your template, this should all be very straightforward. Just fill in the necessary information, highlight all risks, include all necessary caveats, and fire off your report. Execute (run your tests), Decide (based on the results), and Report (and clarify impact).

6. Execute Planned Tests

Execute all applicable tests. This should be very straightforward. You already have a set of tests to run, they have been reviewed, and should be ready for you. Now you follow them. Simple

Log Test Results
You should now log the results of each of the tests that you executed. What passed, what failed, how long each test took, how many bugs were found, etc.

This should be very simple and straightforward – if you have the opportunity, you should automate this step. You have results, so just put them into your results repository (spreadsheet, database, etc.) so that you can build your metrics.

Monitor and Adjust Priorities as Necessary
With every step in your Quality Assurance Procedure, you should be performing continual analysis. This doesn’t need to be hugely time consuming, you just need to be aware of how effective and efficient each of the steps, tests, and reports are.

After logging your test results, you should review them:
• Where are the most bugs occurring?
• What tests are taking the most time?
• Are they taking too much time?
• What areas are not receiving attention?
• What areas have not yet been implemented?
• What areas are solid enough to warrant reduced test scope (at least temporarily) going forward?
• What bugs are impeding testing and not allowing you to test other areas?

7. Deliver Test Results

When you are able to communicate clearly and deliver results the same way every time, then people will trust your reports. When they trust your reports, they will trust your work. This unquestioned credibility makes you more and more effective. Proper Quality Assurance Procedure enforces this style of communication.

Reproducibility and uniformity are key to mastering this. Just like when you go to a McDonalds, you know what to expect, it’s basically going to taste the same wherever you are. This is something that McDonalds’ customers rely on – it is that reliability that brings them back time after time.

You should want to be known for this kind of reliability (if not the taste). Reliable, unquestioned QA is an incredibly valuable commodity. That is why I keep harping on all of these simple steps. These details will make your reports uniform and bring trust and credibility to your work. By implementing and following effective Quality Assurance Procedure you will enhance your reputation, make your work life easier, and increase your efficacy.

Compile Results Using QA End of Build Report Template
Continuing to lead by example and follow proper Quality Assurance Procedure, you grab your End of Build Report template and begin to summarize the results of your testing. List what you did, what happened, and what you will do next.

Translate Results for Project Team
Once you have detailed your test results in your report, add the translation. That is to say, summarize the impact of the results that you are delivering. Communicate in a clear and organized manner and in language that your whole project team can understand.

Avoid useless and extraneous jargon. Don’t use QA Team specific terminology that the developers or producers won’t understand. State “these are the results and here is what they mean.” In implementing effect Quality Assurance Procedure for you team, you can even create a term glossary for use in your report templates. If you team follows your Quality Assurance Procedure as they should, then they will all communicate in the same language every time.

Deliver Results with Translation and Project Status Update
Once you have summarized your test results and quantified their impact, include the project status from QA’s perspective. Now that you have completed this round of testing, what does that mean?

This will depend on what you were able to accomplish during the testing. You may have been able to complete an entire planned testing pass. In that case, your status update might be that QA completed Pre-Alpha Pass #1 and is waiting for next delivery on .

If the build was in such a crippled state that you were only able to perform a fraction of your planned testing, then you could state that QA executed tests “X”, “Y”, and “Z”. You would then want to highlight areas of risk, and state that QA is waiting for the next delivery; which is expected to contain specific upgrades/fixes so that planned testing can be performed without omission, and the project can stay on course. If the delivery does not have those changes, then the project will be at risk (high, moderate, or low…your choice).

Something simple and preventative – the most effective Quality Assurance Procedure). This doesn’t have to be incredibly in depth. You are delivering the results, a succinct summary of what they mean, and the status of the project as seen from QA. Just that simple. Communicating clearly is what makes this preventative – you preclude any chance of misunderstanding when you employ effective Quality Assurance Procedure.

8. Update QA Status

Why is it important to advertise what you are doing now? Because when QA isn’t testing, it is often erroneously thought that QA is sitting on their hands just waiting around. The perception is that you are not being productive.

Squash this (mis)perception by advertising the other ways in which your team adds value to the project and the company. There are many productive actions your team may be taking:
• Updating test suites to match changing design criteria?
• Implementing efficiencies that will result in “X” fewer testing hours per test round
• Resetting your compatibility lab to be ready for the next round of testing
• Reviewing documentation for upcoming projects
• Creating test suites for upcoming projects
• Etc.

This is just a short list. What else does your team do? Whatever it is, make sure that you post it somewhere clearly so that the whole project team (or company) can know the value you add.

9. File Results

Most (if not all) of this data should have already been captured in Step 6 of your Quality Assurance Procedure. Here you are gathering all of that logged data so that it can be copied into a larger repository. Remember, you should be assessing what works best for your Quality Assurance Procedure – if this is not most effective for you, find another way to accomplish this same thing.

Compile Captured Data for Long-Term Analysis
Take the data that you have and put it into your analysis bucket. There are many different ways to analyze your data, but whichever way you choose needs to be in a suitable long-term form. This is so that you can look at today’s data with and against data that you capture in 6 months, 1 year, etc.

So take your data, make sure it is in the format and style that will allow you to analyze it, and put it in your data analyzer (spreadsheet, database, etc.). Now you can look at it and extract useful information from it. Your Quality Assurance Procedure should be implemented in such a way that this is an easy goal to accomplish.

Update Relevant Metrics
Adding your most recent test result data to your ongoing test result analysis repository, you are primed to update your test metrics as needed. You should analyze the newest data on its own (an extension of the monitoring you did in Step 6 of your Quality Assurance Procedure) and then look at it when added to your testing history.

This should give you the information you need to update your testing metrics. You should note any changes to risk areas – have the testing areas that yield the most bugs changed? Has the test time for any area increased or decreased? Has there been a change in the amount of testing time between found bugs?

Whatever metrics you are tracking, use this latest data to compare against individually and analyze as a collective whole. You have the data; this is your chance to use it intelligently. You have followed proper Quality Assurance Procedure so that you could reach this point and enhance your value – seize your opportunity!

Archive data
As I noted above in Step 3 of our Quality Assurance Procedure, you should have a safe and secure location where you will keep all of your QA data. You save your tests, your reports, your templates, and your data, anything useful to this location. You need to have easy access to this data for your ongoing work, but you should be in the habit of archiving (or archiving copies of) all data that you want to guarantee you have access to.

Since you have prepared, tested, analyzed, and reported, now is the time to archive (or archive copies of) everything that you don’t need right in front of you. Everything should be updated so go ahead and archive the important data

10. Prepare for Next Build

This doesn’t need to take long, and probably shouldn’t. Making sure that you have clean, updated copies of your test suites will save you time and headache as you progress through your project.

Here again I am directing you to do the right things first. I realize that this is at the end of a cycle, but this is the time to take this action. And this action is to be taken before the cycle begins again – therefore, right things first.

By preparing your test suites for the next round of testing, you are ensuring your preparedness. Any unforeseen interruptions, scheduling hiccups, or other distractions will be much easier to deal with when you have fewer plates to keep spinning.

So, prepare a clean test suite to use as a base for you next round:
• Delete tests that no longer apply due to design or implementation change
• Add efficiencies found in the previous round of testing
• Edit the test flow so that it is most effective
• Update expected results as appropriate
• Update the revision number and notes
• Etc.

Once you have your squeaky clean, updated test suite ready, save it and move forward

Get QA Team on Same Page
Have a meeting with your team; long or short is up to you. Your goal is to make sure that your entire team is aware of and understands the status of the project.

This is your chance to share reports, point out high-risk areas to each other, and answer any outstanding questions. Any and all changes that have been made to the test suites should be detailed. All implemented efficiencies should be understood and your entire team ready and willing to follow them.

Take a quick breath together and make sure everyone is ready to pursue your common goals.

Prepare Team for Next Round
With everyone on the same page, the final step of this Quality Assurance Procedure is to get your team’s heads ready for the next round. Everyone should know when the next delivery is due. They should know what that build will contain. They should be ready to execute the testing that is planned. And they should know what to do without being told.

Make sure your team is ready. This is a simple investment of just a little time; time that will be well spent when the next build arrives. Ironing out any lingering issues now will pay off handsomely as you move forward.

Each time you take this step and ensure the preparedness of you and your team, you are reinforcing a formidable habit. Every time you do this, you lessen the amount of time and energy you will have to expend on it in the future. Plan to prepare. Work to be ready.

That just creates an opportunity for you to adapt your own system. Regardless of your circumstances, if you choose to succeed, it will be up to you to create the most effective way to show off the service you provide.

Remember to keep learning and keep adapting. Once you have the tools, each situation simply provides opportunities for you to prove why you belong. Show what makes you special!


Return from Quality Assurance Procedure to Successful Quality Assurance Home