• Updates:
• Our Training Courses are now available
• We offer courses that teach Elite QA tester skills and courses that teach you all you need to know about bug writing.
• We even offer the first two courses free! Start your training now!
• In this issue: An Introduction to Finding Bugs
• This month we look at we you can start finding bugs in any program right now. You don’t need a test plan to tell you where to go. We begin you on your journey to understand how to find the weaknesses in any software.
• We give you specific actions to take to find bugs. Learn these principles and apply them to any other testing you ever do – you will impress every time!

The impossible often has a kind of integrity to it which the merely improbable lacks.
~Douglas Adams

How to Find Bugs:
A Primer

Partly this is because testing is so simple. Testing can also be very product-specific. Other than that, becoming a QA tester that understands his/her craft is an experiential journey that each professional really must take on their own.

Now that I’ve said all that, I will attempt to give you a primer on how to find bugs. If you are an experienced QA Professional, chances are you already know a lot of this information. But if you haven’t been testing for years and years, hopefully this will give you some solid ground to understand how to find weaknesses in any software you test.

Following the Script

The product design and technical specification documents establish a product’s requirements. To make sure that each and every requirement behaves as intended, test cases are written to cover each requirement. The outcome of the test cases is then tracked so that the QA team knows if and when all requirements have been tested.

Each test case you execute should give you exact steps to follow. By following the stated steps, you will be verifying the behavior of a feature requirement – or at least one piece of it.

That is why I say that testing is simple – all you have to do is follow directions. This assumes that the test cases you are given have been written clearly. Unfortunately, that is not always the case.

If the test cases you are trying to execute are not crystal clear, make sure you receive clarification as to what the intended testing is to cover. You can either update the test case yourself (so it is easier to follow in the future) or get it on the to-do list of whoever is responsible for that.

Test cases must be clear and accurate or your testing will suffer. You may not test the right thing; you may miss requirements that need to be validated – in short, get your tests accurate or risk participating in bad QA.

While executing test cases, take each one a step at a time. Perform each step of each test with diligence and attention. Make sure you aware of the outcome of every move you make. Compare the results you witness with the expected results as stated in each test case. Anytime there is not a match, you have found a bug!

But what if you don’t have a test case in front of you? What if you have been tasked with performing exploratory/ad-hoc/real-world/end-user testing? How will you be able to find bugs?

Follow me…

Low-Hanging Fruit

Where should you begin your exploratory testing? How can you make an impression on your boss and show that you have a knack for finding weaknesses in software? Being able to quickly and effectively identify where a product is weak, is really the art of software testing.

Improving your skill and developing your craft take time. But when you become adept at the art of testing, software you have never touched before will bend to your will.

Programs will stand in line to spring open and share their secrets with you. You will have more bugs than you can possibly log in a single day. You will be a known assassin and you will be in demand.

But, until you develop the feel yourself, here are some ideas of where to look.

Common software weak points:

• Data Capture
• Selectors
• ReadMe.txt
• Transitions
• Interrrupts
• Defaults
• Un-Install

Data Capture
Any time a program or web page have data capture fields, you have a chance to exploit them. Data captured goes into databases for future use so if you can affect the data capture field, you may be able to affect the database.

Most data entry fields allow you to enter whatever you want. Their rule checking happens upon submission. Anytime you can get around the rules that the database on the back end needs enforced so that it can do its job, you can put the whole system in jeopardy. So here are some things to try:

• How many characters can you type in a field?
• This is usually limited to 256 or less. So, if they limit the amount you can type in, then go find a copy of War and Peace and see if you can Copy and Paste. Often you can paste more into a field than you can type
• Which characters can you enter by typing? Many programs don't like "special characters" (like: @, #, %, $, and &)
• Again this is often limited, so go back to Copy and Paste. Now which characters can you enter into the field?
• If there are any rules stated next to the field (minimum number of characters, etc.) see if they are being enforced
• If there are multiple data capture/entry fields, use the [Tab] key to navigate from one to the next. See if they go from one to the next in a logical order

Selectors
Much like data capture/entry fields, selectors can be very weak in the early stages of development. This is often due to the development team using non-standard, basic “rules” that they expect to work. It is thought to be such mundane work, and thus overlooked, that this is often a rich area for bugs.

Radio buttons, Drop-down menus, or any other selection mechanism should be toyed with as though you were a halfway intelligent monkey.

• Left and right mouse click very quickly on a selection
• Double-click a selection
• (Very)Quickly click one and then another selection
• Click and drag prior to selection
• Select any blank line item option

The other thing to check is what happens if you don’t select any of the available options. Are there defaults selected already? Are the defaults functional even if you don’t select them?

ReadMe.txt
R.T.F.M. = Read the Manual (Don’t ask what the “F” stands for)

Part of your job as a QA tester is to ensure that the information made available to the user is accurate. It is very common to use an old ReadMe.txt file on a new product or updated product version. Often the ReadMe is updated very late in the project, but it will have to be (or should be) updated prior to launch.

Get into the ReadMe immediately and see if it is accurate. You can often find several bugs to log just in outdated information, information that wasn’t updated prior to the previous release, or even copyrights. In the real world, not many users actually R.T.F.M., but if you have a ReadMe that is shipping with your product, then it had better be accurate.

Transitions
Programs are often very weak during any and all transitions. Transitions from one screen to another, from one scene to another, from one user state to another, or from one game level or state to another. Make selections that will engage the program during a transition.

This means, make a selection (whether or not you can see it onscreen) while a transition is occurring. Even if you can’t see the selection you are making, click on the location of where selector would be normally.

Interrrupts
Just as stated above with transitions, interrupting the program while it is trying to do something can often give it problems. This is especially true in the earlier stages of development (hopefully this becomes less common later in the project).

Create an interrupt while the program is not “expecting” one:

• While it is loading
• While reading information
• While writing information
• One example would be to unplug a memory stick (flash drive, USB drive) from a computer while it is reading/writing – bad things happen. Apply this test to devices and memory reads/writes other than external USB, that is only an example so you understand the principle

Defaults
Verify that a program’s default selections function properly if you don’t select anything.

Another issue is to install to a non-default location. There are thousands of programs on the market right now that do not function properly if they are installed anywhere other than the default (sad, but true).

Even if the main body of the program works, check the peripherals. Do the installed shortcuts recognize that the program was installed to a non-default location or is the shortcut path hard-coded? Does the program save information to a non-default location?

Un-Install
UnInstall the program and see if it really does “Un-Install”. Upon completion of the Un-Installation, does the program inform the user that everything has been uninstalled (or does the program imply it)?

If, upon completion, the program tells the user what was not removed – OR – that not everything was removed and perhaps tells the user what was left and/or where, then that is ok (technically – although it is always feels like amateur hour to me). But if the user is led to believe that everything has been uninstalled, this should be true.

Often, however, a program will tell the user (or strongly imply) that everything has been removed when this is simply not the case. This can get the company a great deal of bad press depending on what is left behind. News travels fast on the internet.

Run a standard search on all hard drives for the name of the program, shortcuts, etc., See what is left. Search for directories that the program installs; save information, preferences, etc.

If you are on a PC you can also run “regedit” and search the registry for program remnants (from the [Start] menu, select “Run” – and then type “regedit” without the quotes, then select [Ok] or press [Enter] – your registry will open. F3 will launch search function). If a program tells the user all has been removed, then all really should be removed.

And don’t forget to combine Non-Default installation and full Un-Installation. Make sure a program can remove itself after installing somewhere it is not expected to.

And You’re Off!

By practicing tests that are known to reveal bugs is a very quick way to get a feel for software testing. Execute effective tests and learn what they do. This is your shortcut to understanding the art of software testing.

I'd take the awe of understanding over the awe of ignorance any day.
~Douglas Adams