Tag Archives : test quality


An example of how TestLess’ TestDetectors work

In this post, I’ll illustrate the workflow that you can use to extend TestLess so it can analyze any test suite. I will use a real example from a real customer to show you how easy it is to extend TestLess.     The Problem Recently, I received the following question from a customer:   I’d […]


Mutator and TestLess are improving the quality of software you use!

In this post we take a quick look back at the success stories for Mutator and TestLess that you might not know about:   Bloom Filters Bloom filters are everywhere nowadays, and having seen the quality improvements that Mutator started, it is no wonder that the authors of one popular implementation decided to continue on that road.   Best of Breed Mutator […]


How well is it tested: Puppet (part 2 of 2)

In the last post, I showed analysis results of Puppet’s test duplication along with other interesting findings. The relationships discovered seem to suggest that test duplication is not due to chance alone, and that there is a complex relationship in terms of amount of tests in the suites that is not entirely linear. The results in the post also […]

Test duplication vs mutation score (log-scaled and reversed)

How well is it tested: Puppet (part 1 of 2)

Puppet is a “Server automation framework and application” that it utilized by people worldwide and even Fortune 100 companies (according to Puppet’s website) to manage part of their devops work. It is also one of the key projects that “powers Github.” Puppet is written in Ruby and unit-tested using the BDD framework Rspec. Given Puppet’s popularity and utility, […]


TestLess now supports nested contexts!

BDD-style testers: Rejoice! One of the super helpful aspects of BDD test frameworks is their utilization of nested contexts. Nested contexts not only allow test writers to group different tests, but also allow you to delineate their state into different sets in order to separate permutations of their SUT’s (system under test) behavior into logical groups. This separation and […]


Mutator now supports multiple source files!

You asked, we listened! We heard you loud and clear and we are thrilled to introduce multi-source support for Mutator! Previously, Mutator only allowed one source file per test suite using the assumption that each test suite had one and only one source (a.k.a. system under test, or SUT). This original design choice was deliberately made partly to encourage […]


How well is it tested: Composer

Composer is a very popular dependency manager that many PHP projects use. With the newly introduced support for PHP by Mutator, I set off to rate the quality of Composer. For the analysis, I utilized the Composer snapshot of version 1.0-dev, from Aug 24 2015. Recall that the mutation score of a test suite is a quality metric […]

95% confidence interval for Composer's test quality