What do I need to know in order to understand Data Mining?

I just had a job interview that went well for a qa tester position. The Manager stated that they are just starting up after a merger and need help with the qa process, and such (and I've had experience there).

The real concern I have is that most of the work will be with Databases and data mining (I'm not really familiar with). They just started using HP Quality Center (I have very little experience with) and need people that are willing to learn (that's me!).

So how can I best understand data mining and the technique required to be successful at this job...looks like I will be hired.

Thank you,
rustymanualqatester

********************************************

Hi Rusty,

Hopefully, you did get hired!

Data mining is the sexy term used in the industry to describe gathering, sorting, and (hopefully) making sense of information. As databases have become larger and more capable of holding more and more diverse information, the need to sort, soft, and quantify that information increased equally. A company that learns to perform effective data mining can, just like an infant that learns object permanence, advance to the next stage in their development.

I haven’t used Quality Center myself, but the HP tools are pretty robust. They should be fairly straightforward in their use and operation. I am certain they have an extensive Help section, if you need a quick query.

The actual gathering of the data is usually the simple part. In most data gathering tools, commands are often entered via a WYSIWYG interface or a very straightforward simple-script entry interface. If you are using a simple script language then often you will have a list of available commands that you can use with a list (or couple of lists) of variables/switches. Either way, there should be a fairly straightforward and (hopefully) finite number of options for you when you begin. If you are an eager student, you should be able to become quite proficient in no time

The real key in Data mining is in knowing what question(s) you are trying to answer before you begin. Usually there is so much information that if you don’t have a specific focus going in, you end up with loads of information that you won’t need…and that just makes your job harder!

So before you begin, find out what information is being sought. Data mining is a way of building metrics, determining what has happened (and therefore may happen), and quantifying your (or someone else’s) efficacy. But the only way you can gather the right data is by knowing what information is desired. If you’re lucky, you may even be told which specific files/file type/date range information to gather.

Of course, the ease with which you are able to do all of this is also dependent on the way that the information was stored initially. If the information you need is clumped together in a field with 50 other variables, then you will have to parse through more information to get a dataset that is of use to you. If this is the case, then just eliminate subsets of variables one (or more if you are confident) at a time.

Rules to Live By:

ALWAYS have a backup

Maintain your backup in a separate location (NOT on the same hard drive or computer)

Name your backup something different than the file(s) you are working with

Gather…Sort…Analyze…Quantify