Tag Archives : Ruby


How well is it tested: Puppet (part 2 of 2)

In the last post, I showed analysis results of Puppet’s test duplication along with other interesting findings. The relationships discovered seem to suggest that test duplication is not due to chance alone, and that there is a complex relationship in terms of amount of tests in the suites that is not entirely linear. The results in the post also […]

Test duplication vs mutation score (log-scaled and reversed)

How well is it tested: Puppet (part 1 of 2)

Puppet is a “Server automation framework and application” that it utilized by people worldwide and even Fortune 100 companies (according to Puppet’s website) to manage part of their devops work. It is also one of the key projects that “powers Github.” Puppet is written in Ruby and unit-tested using the BDD framework Rspec. Given Puppet’s popularity and utility, […]


How well is it tested: Mutant

This time, Mutator analyzes another mutation testing tool for Ruby called Mutant. Mutant is an alternative to mutation analysis for Ruby projects. It is fast because it applies mutations on the abstract syntax tree. However, Mutant has several disadvantages. First, since it goes under the Ruby hood, it only supports Ruby exclusively and, second, it only support […]

Mutant's test quality

How well is it tested: AI4R

This time we apply Mutator to Artificial Intelligence for Ruby (AI4R) to analyze how well it has been tested.   Basics Mutator analyzed AI4R version 1.13 which has 30 test suites and 35 core source files. With the exception of one outlier (a failing test suite, see below), all test suites were selected for analysis. Since practically all […]