So you want to start doing TDD

I have been practicing TDD for about three years now. The process of starting up on this discipline is fairly fresh in my memory, so I decided to write this post to help those who are just about to go through the same thing.

TDD is one of those things that are easy to learn but difficult to master. It is fairly easy to start going through the TDD cycle and produce a lot of tests. It does not take you many months to get the hang of that part. However, you will soon learn that TDD is not just writing a lot of tests. The tests are part of your code base too. They need to be maintained as well, must be just as well written as the product code. Learning how to write good and maintainable tests takes time and practice. The purpose of this blog post is in part to help you speed this process up a little bit, so you won’t have time to write quite as many crappy, unmaintainable tests. Take a look at this informal non-scientific illustration. It shows the number of tests I write and their quality, as I become increasingly proficient at writing code with TDD:

The chart shows that as you start out on TDD, you quickly end up writing a lot of tests. That part of TDD is easy to learn. However, there are problem here: The tests are hard to write, so they slow you down. They are brittle, so your code becomes harder to change. They are hard to read, so maintaining them slows you down too. And they are slow to run, increasing your build times. You have reached the Chasm of the Many Crappy Tests, but don’t worry. If you stick to it, keep learning and don’t give up, then you will eventually cross it and reach TDD nirvana (or at least it won’t suck so bad anymore).

Roy Osherove says that good tests have three interlocking properties: They are readable, trustworthy and maintainable. A fourth property, fast, is sometimes added to the mix, but it is merely a useful property, not an essential one.

Readability

A readable test is one that reveals its purpose or reason for being. Essentially what the test is testing. A lot of readability can relatively easily be bought simply by giving the test a proper name. If you are testing a queue, for instance, then don’t have a test called “testPoll1” — this is a silly name that reveals little other than the “poll” method might be called somewhere. Instead, name your tests after useful observable behaviour that the code must exhibit. For instance, “itemPushedOntoEmptyQueueMustBePollable” is a name that describes some useful observable behaviour about queues: you must be able to poll an item from a queue, that has been pushed onto the queue.

I consider the names of the tests to be part of the informal specification of the behaviour of the unit. When I have to implement a new class, I often start in its test case by writing to-do comments with names for each of the tests I want to write. I use these names as a sort of up-front design for the unit — an initial draft of the specification for the unit in question.

When your tests are named after useful observable behaviour, you naturally end up only testing for one thing in each test. It is acceptable to have more than one assertion, as long as you only assert on one thing; most likely a single object.

Finding the correct balance between having set-up code inside the tests, or factory methods or a dedicated set-up method, is also an important element of readability. You want to reduce the amount of code in the tests, but you also want it to be plainly clear what the test is doing. It is easy to get into the pitfall of hiding too many details in set-up or factory methods, so a reader have to hunt these methods down if he want to make sense of your test. The Don’t Repeat Yourself principle, DRY for short, is often hammered pretty hard into the brains of good programmers. However, it is perfectly acceptable for test code to be a little “humid.”

Trust

A trustworthy test is one that deterministically fails or passes. Tests that depend on your machine being configured properly, or any other kind of external variable, are not trustworthy, because you don’t know if failure means that your machine is misconfigured, or if the code is buggy.

These tests that depend on external variables are integration tests, and they should put in a separate project along with some documentation on how to get them up and running. Then their slow run times also won’t affect your normal build times.

An external variable is anything that you don’t have complete control over: Files, databases, time, 3rd party code. With regards to time, just create your DateTime instances with a fixed instant rather than the fleeting “now” and call it a day. In a unit test, you want to use the same exact test data every time, but if your test depend on the value of “now,” then you will effectively get a different test every time you run it. As for threads, Roy says these are external variables as well, and I guess they technically are, but I’m still not convinced that they are integration tests (then again, I might be special in that regard).

Maintainability

A maintainable test is one that does not easily break when you maintain it. Loose coupling is probably the single biggest contributor to maintainability. Factory methods decouple you from constructors, that tend to have their parameter lists changed more often than other methods. Only test a unit through its public API, and “protected” is effectively public. When you do this, you tend to be more decoupled from the implementation details of that unit. Abstractions can still leak out, though, but this is a design challenge that you should tackle in your product code.

Avoid putting logic in your test code; things like if-statements, loops and switch-case statements. Where there is logic, there is a potential for bugs, and you don’t want bugs in your tests — especially not long lasting ones. Also avoid magic numbers; you want to be able to tell why a certain value is passed in as a parameter, or why a certain value is returned from a method. Don’t calculate the expected value, because you could end up mirroring the product code, including any bugs it might have. Also don’t share state between tests — they must be runnable in any order — or run a test from within another test. Keep them isolated.

Giving tests meaningful names is also important for maintainability, as well as readability. When you can infer from the name what the test is trying to verify, then the test code can be checked to see that it is actually testing what is says it is testing. You can make sure that it keeps testing for the same behaviour, even when there are changes to the API it is using. It can also be rewritten if the code is a complete mess.

In following the SOLID principles, or the old virtues of loose coupling and high cohesion, you will typically end up with units that do just one thing. However, what “one thing” is depends on your level of abstraction, and the unit may also have to operate in a number of different scenarios. These factors can complicate the set-up code, and complicated set-up code is a maintainability pain. A way to deal with this, is to have multiple test cases, or multiple test fixtures, for a unit — one for each scenario. Then each scenario only need set-up code that is relevant for that particular scenario, and you end up with less clutter in the set-up code. Since many test-bugs are in the set-up code, you will most likely also end up with more trustworthy tests.

When your tests are readable, then they become easier to maintain. When the tests are maintainable, then chances are that they will actually be maintained. When you know that the tests are maintained, and you can tell what they are testing, then you can trust them to be that safety net they are supposed to be.

Summary

• Start out by writing many tests and get the hang of the TDD rhythm.
• Gradually improve test quality — read blogs, books or other people code.
• Stick to it, don’t give up. You will end up a better programmer overall.
• Test behaviour rather than methods, and only through the public API.
• Give your tests descriptive names. This is essential for readability.
• Favour “humid” tests, when you choose where to place set-up code.

And lastly, if you find that some part of your code is particularly difficult to test, then chances are that you are being challenged to come up with a more testable design. Testability often means looser coupling and higher cohesion, so it tends to be a good idea to listen to your tests.

Good luck with it!