Foundation Series: Unit Testing of Software

What are unit tests?

Testing software is more than just manually banging around (also called monkey testing) and trying to break different parts of the software application. Unit testing is testing a subset of the functionality of a piece of software. A unit test is different from a system test in that it provides information only about a particular subset of the software. In our previous Foundation series post on black box and white box testing, we used the inspections that come bundled with an oil change as examples of unit tests.

Unit tests don’t show us the whole picture.

A unit test only tells us about a specific piece of information. When working with a client who’s company makes telephone switches, who’s internal software development team did not use unit tests we discussed the following analogy:
Unit tests let us see very specific information, but not all of the information. Unit tests might show us the following:

A bell that makes a nice sound when ringing.

A dial that lets us enter numbers.

A horn that lets us listen to information.

We learn a lot about the system from these “pictures” that the unit tests give us, but we don’t learn everything about the system.

We knew (ahead of time) that we were inspecting a phone, and with our “unit tests” we now know that we can dial a phone number, listen to the person on the other end of the line, and hear when the phone is ringing. Since we know about phones, we realize that we aren’t “testing” everything. We don’t know if the phone can process sounds originating at our end. We don’t know if the phone will transmit signals back and forth to other phones. We don’t know if it is attached to the wall in a sturdy fashion.

Unit testing doesn’t seem like such a good idea – there’s so much we need to know that these unit tests don’t tell us. There are two approaches we can take. The first is to combine our unit tests with system tests which inspect the entire system – also called end to end tests. The second is to create enough unit tests to inspect all of the important aspects. With enough unit tests, we can characterize the system (and know that it is a working phone that meets all of our requirements).

Software developers can identify which parts of their software need to be tested. In fact, this is a key principal of testing-driven development (TDD) – identify the tests, then write the code. When the tests pass, the code is done.

Why not use system tests?

The system test inspects (or at least exercises) everything in the software. It gives us a big picture view. Ultimately, our stakeholders care about one thing – does the software work? And for them, that means everything has to work. The intuitive way to test, then, is to have tests that test everything. System testing is also known as functional testing.

These comprehensive tests tell us everything we want to know. Why don’t we use them?

There is a downside to system testing. In the long run, it’s more expensive than unit testing. But the right way to approach continuous integration is to do both kinds of testing.

In our Software testing series post on blackbox and whitebox testing we discuss several tradeoffs associated with the different types of testing. For most organizations, the best answer is to do both kinds of testing – do some of each. This is known as greybox testing, or grey box testing.

System tests are more expensive, because they are more brittle and require more maintenance effort to keep the tests running. The more your software changes, the faster these costs add up. Furthermore, with Agile practices, where portions of the system are built and tested incrementally, with changes along the way, system tests can be debilitatingly expensive to maintain.

Because unit tests only inspect a subset of the software, they only incur maintenance costs when that subset is modified. Unit testing is done by the developers, who write tests to assure that sections of the software behave as designed. This is different from functional testing, that assures that the overall software meets the requirements.
There are more articles on software testing in our software testing series.
– – –

Check out the index of the Foundation series posts for other introductory articles.