Caution when using TDD’s Triangluation technique


If you are a TDD (Test-Driven Development) practitioner, you will most likely find the triangulation technique (a.k.a specific/generic cycle) incredibly useful for driving the design of your software. In short, triangulation is a technique that uses tests as a guide to help your software implementation emerge, little by little, from requirements.

I like to think of triangulation akin to a way a sculptor works with stone or marble: the stone is slowly chipped and cut here and there, carefully sculpted with specialized tools, sanded, and frequently inspected in different places, where the craftsman’s (or woman’s) trained eyes determine where the slab needs work, thus continuing until the final piece emerges from the stone. Similarly, the software engineer drives the design and implementation of software with the help of carefully crafted tests, molding the implementation from the specific all the way up to a general level, carefully creating and leveraging abstractions, until the complete implementation emerges, representing the final design and fulfilling the original requirements.

As you can imagine, triangulation is truly a craftsman’s technique that requires care, attention to detail and a trained “craftsman’s” eye. As such, I find that triangulation is a great way to drive an implementation not only because it is an “emergent” technique that helps implement software in an intuitive and organic approach, but also because the resulting implementation is naturally extremely well tested.

Recently, I read a blog post detailing the usage and benefits of this technique. A funny aspect of all blog posts written about triangulation, including the one just mentioned, is that they never tell the whole story. They only expose the good but either hide or completely ignore the bad aspects (assuming they even know about the bad aspects). Such blog posts therefore leave the audience thinking that the technique is a silver bullet for their software implementation and design woes. Kind of like a “No batteries needed, runs with free energy! sales pitch that comes back to bite later… and bite hard.

Unfortunately, like all things in life, there is no such thing as a free lunch, and triangulation has a negative side-effect that you should be aware of: duplication of tests, sometimes extreme.

Let’s analyze the final implementation from the said blog post to illustrate.

In terms of quality, it is no surprise that the tests completely cover the implementation. In other words, Mutator shows that the tests are superb and achieve adequate mutation score of 100%.

 

Triangulation yields well-tested software

Triangulation yields well-tested software

 

This is the good side of TDD in general, and triangulation in particular, for implementing software. In other words, assuming you apply triangulation correctly and diligently, it assures you that your code will be completely covered by at least one test.

But this is also the downside, because the more your triangulate, the more tests you create.

For example: in the final implementation that we are analyzing, we find that 72% of the tests are totally redundant!

 

Triangulation yields highly-redundant tests

Triangulation yields highly-redundant tests

 

That’s right! The amount of tests that are truly necessary to achieve 100% mutation score and cover the entire implementation is only 5 (five) out of 18. Viewed differently: practically only 1 (one) out of every 4 tests that were created using triangulation are necessary, while all others can be removed.

You can see now that triangulation helps your software implementations emerge but at the expense of test redundancy, to the point that your once-manageable and simple test suite immediately becomes an unmanageable mess of redundancy.

Is test redundancy a problem?

Yes! For several reasons:

First, test writing is not free. Writing tests requires work and time. Hence, by duplicating tests you are actually spending more time than needed and you are unnecessarily duplicating your workload.

Second, doing redundant work naturally means that you will be spending more time and money than you would otherwise, possibly causing your schedule to be overrun and your budget to be drained.

Third, having more tests (whether redundant or not) means that your tests run longer. However, having high amounts of tests that are totally redundant means that you unnecessarily spend high amounts of time waiting for results that you could achieve much sooner.

Fourth, it might not be as easy to visually spot redundant tests as you might have expected. Some tests seem like they are testing completely different areas, while others seem to be covering other tests but only in certain areas. It might therefore be very difficult to determine which tests to leave and which ones to remove even when relying on code coverage metrics. Instead, I rely on Mutator and TestLess to manage my tests.

So as you can see, even though the link between test duplication and low quality software disappears (fortunately) in triangulated code, the side-effects of using the technique still need to be weighted and carefully considered against other software design/implementation options.

You don’t have to take my word (or the evidence above) either. Simply listen to Kent Beck, who postulated this side-effect in his book “Test-Driven Development by example” (emphasis mine):

 

Although they seem simple, the rules for Triangulation create an infinite loop. Once we have the two assertions and we have abstracted the correct implementation […], we can delete one of the assertions on the grounds that it is completely redundant with the other.

 

Compared to the typical methodologies that seem to be common in open-source software, triangulation is interestingly at the opposite end of the spectrum in terms of test duplication and test quality. In other words, whereas contributors to open source tend to create very little, highly redundant but low quality tests in their process of implementing software, triangulation yields extremely well-tested software at the expense of too many tests and too many duplicates.

Choose wisely.

 

 

 

 


2 thoughts on “Caution when using TDD’s Triangluation technique

Comments are closed.