Software product success.
This is the second in a three-part post about using tags as a means to organize an automated test suite.
Part 2 of this post can be read as a standalone article. If it were, it would be titled Top five problems with test automation suites. If you’re only reading this post and not parts 1 and 3, pretend that this is the title.
This post is a follow-up to our previous case study on incorporating unit testing into an existing team’s development environment. The case study is based on a real solution that has already started reaping rewards for our client, and is gaining momentum. We’re now looking at making it easier for the development team to maintain this test suite, and proposing some extensions – including a form of tagging.
This post is a test automation case study, but at a higher level.
We’ll talk about it in terms of defining the problem, and then discuss the objective (what we proposed to do to solve the problem), the strategy (how we went about doing it) and the tactics (how we executed the strategy). Since this happened in the real world, we’ll also identify the constraints within which we had to operate.
Testing software is more than just manually banging around (also called monkey testing) and trying to break different parts of the software application. Unit testing is testing a subset of the functionality of a piece of software. A unit test is different from a system test in that it provides information only about a particular subset of the software. In our previous Foundation series post on black box and white box testing, we used the inspections that come bundled with an oil change as examples of unit tests.
Should I use black box testing or white box testing for my software?
You will hear three answers to this question – black, white, and gray. We recently published a foundation series post on black box and white box testing – which serves as a good background document. We also mention greybox (or gray box) testing as a layered approach to combining both disciplines.
Given those definitions, let’s look at the pros and cons of each style of testing.
Blackbox tests and whitebox tests.
These terms get thrown about quite a bit. In a previous post, we referenced Marc Clifton’s advanced unit testing series. If you were already familiar with the domain, his article could immediately build on that background knowledge and extend it.
Software testing can be most simply described as “for a given set of inputs into a software application, evaluate a set of outputs.” Software testing is a cause-and-effect analysis.
In a previous post, we mentioned a link to the first in a series of articles by Marc Clifton at The Code Project, a .NET resource site. Here are the links to all 5 of the articles in Marc’s series.