10 Skills of Elite Testers: Day #5

There are few things that you can spend your time on that will yield more information about the state of your projects than your bug database (commonly known as a bugbase). Therefore it is your assignment to master it! Being able to use your bugbase effectively will be critical in being able to show your true worth to your team and your company.

Skill #7:
Master Your Bugbase

The sweat of hard work is not to be displayed. It is much more graceful to appear favored by the gods.
~Maxine Hong Kingston

• Comprehensive knowledge of all the issues preventing your product from shipping and their status
• The ability to enhance your reports with bug statistics
• Prevent you and your team from entering duplicate bugs
• Decrease the time you spend searching for issues to determine if they have been previously entered
• The ability to separate yourself from the average tester by doing all of the above

In my years working in Software Quality Assurance I have used no fewer than 15 different bugbases. I have used FileMaker 3.0, 4.0, 5.0, DevTrack, TestTrack, Bugzilla, FogBugz, and Jira to name just a few.

The principles are always the same, only the setup and search features really differ. Below I will give you a few simple tips that, once mastered, will make you king (or queen) of your bugbase and have people coming to you to help them massage it as only you can.

Review Bugs Daily

Your first step is to spend 30 minutes each morning before you begin testing, reviewing the Active bugs. If you do this, you will know them all in no time.

Where do you find these 30 minutes?
Either speak to your Lead and tell them what you are trying to accomplish (having a clear understanding of all the Active issues logged against the project so that you can be a more useful member of the team) and assure them that this will not take more than 30 minutes each morning, or get to work early.

This is simple, but it is all on you. Get to work before you are expected to be working on anything else and study the bugbase for 30 minutes. For most effective use of your time:

• Begin by reviewing the most recently entered Active bugs and work your way backward in time by date entered. This will get you up to speed on the issues that are currently assigned to be fixed in the current release. You will quickly learn where the program is the weakest, what issues have not yet been addressed, and which bugs are being given what Priority level. This will give you the informed opportunity to spend your testing time the most effectively and efficiently. You will not spend time testing areas that have been completely covered or have no weaknesses in the code (as if that were possible); you will know how to attack the program to unearth the greatest vulnerabilities.
• Review the highest priority bugs first and work your way down the prioritization ladder
• Thus review all of the Priority 1 bugs first from most current to oldest
• Then review all of the Priority 2 bugs from most current to oldest
• Select the Priority 3 bugs…lather...rinse…repeat
• Once you have completed the review of the Active bugs, move onto any Deferred bugs. By having and understanding of what bugs have already been Deferred, you will have insight into the issues that are slated for a subsequent release. You will know what features are not considered a priority in the current release and can minimize the time you spend exploiting those known weaknesses. Or if you feel strongly that a Deferred bug should be addressed, you will have the information you need to be proactive and show in what other ways that Deferred bug will adversely affect the end-user.
• Review all Priority 1 and Priority 2 bugs that have already been deferred (if any)
• Then move onto the Closed bugs. Once have inventoried the Closed bugs, you will have a more complete history of the project. You will see what bugs have been logged and fixed and can see what the fix was. This should help enhance your ability to effectively test the program going forward since you will have researched the lifecycle of the bugs that have already been logged and addressed.
• Review Priority 1 and Priority 2 bugs that have been closed within the last month or go back 100 bugs, whichever comes first. Don’t go back any farther than that during this exercise – this will get you the information you need to get started
• Pay attention to what the fix was (this will help you understand the current behavior of the program and give you an idea of how your development team addresses bugs)

Once completed, you should be up to date with all of the bugs in the bugbase. You should have a high-level understanding of the major issues that have been logged against the project so far.

This work may take you only a single session or it may take several days to complete. Your goal is to be up to date. How long this takes depends on how many bugs were logged and fixed prior to your inclusion in the project. Once completed, you will only be performing maintenance each day. The hard work will be behind you.

Your next assignment is to set up two searches (or filters) that you will run every day. These two searches are:

• Bugs Opened today
• Include all bugs that are newly Opened or that were Closed but got Re-Opened today
• Bugs Closed today
  • This includes (you guessed it) all bugs Closed today

Set these searches up in your bugbase and save them so that you can run them every day. Then make sure you run them each and every day. Run the searches and review all of the resulting bugs. Since you are up to speed on all the Active issues, this shouldn’t take you more than 15 minutes each day.

Is that worth it to you? It should be. Spending 15 minutes a day to know the Status of all bugs that affect the project is one of the highest return activities you can invest in for yourself as a Quality Assurance Professional.

With the knowledge you gain from this daily review, you will:

• Be able to speak accurately to questions about any bug affecting your project
• Know when you encounter a bug whether it has already been logged or not – thus preventing you (and your team) from entering duplicate issues
• Have an insider’s view into how issues are prioritized and addressed during a project
• Be able to focus your testing on what you know are the weakest areas of the program
• Have the ability to compile your reports with detailed bug metrics much quicker

With this knowledge and these abilities, you will:

• Be viewed as a go-to person on the project
• Increase your productivity by spending your time testing more effectively and efficiently
• Improve the overall performance of your team, thus making your Lead and Manager look better in the eyes of their bosses
• Deliver reports that will show what an outstanding performer you are
• Be seen as an asset that should be developed so that the company will reap even greater benefits by employing you

Master the Search Functions

Some allow you to enter multiple search criteria at once for a single query that will yield the results you desire. Others will require you to learn the filtering system that the bugbase uses. If this is the case, you will need to understand what each filter selection that you enter will yield. Fortunately all bugbases have a help feature or link that you can use to familiarize yourself with its functionality.

Read through the Help provided by the bugbase to get an understanding of how its search and filtering capabilities work. Explore these features and practice until you are able to create searches with the following parameters:

• Active Bugs:
• Opened by a specific tester
• Assigned to a specific tester or developer
• Opened during a designated timeframe
• Opened due to testing a specific section of the program
• Opened due to executing specific portions of your test suites
• Opened by software (the program you are testing) version number
• Resolved Bugs:
• Resolved by version number
• Sorted by Resolution Type (Fixed, Deferred, Will Not fix, etc.)
• Resolved within a designated timeframe
• Closed Bugs:
• Closed by a specific tester
• Closed within a designated timeframe
• Closed by version number
• Sorted by test suite section
• Deferred Bugs:
• Deferred by version number
• Deferred within a designated timeframe
• Sorted by test suite section

Practice until you are able to get your bugbase to give you each of the results listed above. Once you do, save those searches (if possible) so that you don’t have to remember how to do the search each and every time.

Having these searches readily available to you will give you the ability to:

• Be fully aware of the Status of your project at all times
• Know if you are testing a build that is particularly feeble or robust
• Track your bug finding, bug logging, and fix rate
• Answer questions about the Status of all bugs related to the project when anyone asks
• Give you the data you will want to include in your status reports to show off how valuable you are to the project and to your team

Learn the Export Feature

When someone sees a number like 48%, they know that they are looking at a number that represents almost half, right? But do they really see the impact of what that means?

When you are able to combine your hard-data numbers along with a pretty little graph to illustrate what those numbers mean in relation to the whole, it is much more impactful to your audience. Export data from the bugbase showing what sort of impact you had in each phase of the project .

Create a graph that shows:

• How many bugs were entered total – this gives the baseline that you will be measuring against
• How many bugs you entered
• Of the bugs you entered, how many were fixed – this shows that the bugs you entered were of high enough severity and priority, and were written well enough to quickly be addressed

Can you see how being able to actually show your boss how many bugs you have found, what percentage they represent of the total, and the fix rate of your bugs would have an immediate impact in showing off your worth to the team?

You can show this as a chart, or a line graph, or a pie chart, or whatever style you like. The key is to be sure that it is comprehensible to those you will show it off to.

Which do you think would give you a better chance of getting a raise or of being promoted?

• “Look boss, I work hard here every day and really help out the team. I find great bugs! Just ask anyone, they’ll tell you.”

• “In the last 5 weeks on this project I’ve been responsible for logging 43% of the bugs that have been found. Of those, 74% were Priority 1, and to date 87% of them have been fixed. I find and log more bugs per hour spent testing than any other tester on the project.” And as you say this, you are showing your boss each graph/chart that you have prepared from your searches

Do you see how the second scenario would give you a better chance to impress? You are using the numbers that are available to you and backing up everything you say with hard evidence. You boss may question you, but if you have done your homework correctly all of your data will be accurate.

Keep in mind that your boss will probably simply be impressed that you are able to present your arguments in this way. This is how QA should present their data; show the numbers.

Your boss will be impressed, as well he/she should be. You should now be top of the list for more money, responsibility, promotions, and/or development by the company.

See? Numbers are your friend.

Great ability develops and reveals itself increasingly with every new assignment.
~Baltasar Gracian