Testability
TESTABILITY
| TTM | ROI | Sellability | Agility | Reputation |
The ability to easily test software (its Testability) - and be confident in its outcome - is a critical aspect of software engineering. Without it Stakeholder Confidence wanes, release complexity increases, deliveries slow, and Innovation is impeded.
WHY TEST?
In this section (Why Test?), I describe why we test software, and consequently, the importance of testability.
The stock answer is that testing (and testability) verifies quality (e.g. product quality), protecting other important characteristics like customer satisfaction (and Reputation). But testing can also:
- Increase Stakeholder Confidence.
- Stimulate Innovation.
- Support staff retention.
- Improve TTM.
- Improve longer-term ROI.
- Protect Reputation.
Manual testing is often a lengthy and costly undertaking, and carries the following risks:
- It can impede quality and innovation.
- It's easy to miss something and therefore to release untested software.
- It has delivery risk and capacity management challenges; i.e. is there a tester available, with the necessary experience, when needed?
We should, in most cases, prefer automation - something that testability promotes. Automation can greatly enhance delivery speed (at least after the initial effort), promote (sustained) developer efficiency, increase the overall quality of a product, enable us to Learn Fast, and support Innovation (through the use of a Safety Net). We can also use alternative development approaches, such as Test-Driven Development (TDD), to drive our implementation forward, through tests.
A SAFETY NET?
We need a Safety Net in order to improve and evolve, enabling staff to solve problems in more unique and innovative ways, using outlier techniques that are highly innovative, but without facing great risk.
TESTABILITY DETERMINATION
Testability should be a determination of the necessity, lifetime, and the perceived change lifecycle of a feature / product. A feature that is critical to success, enduring, and is “change volatile” (faces repeated changes) is typically in need of high testability and testability should be prioritised there.
PILLARS AFFECTED
Testability affects (and is affected by) all five business pillars.
TTM
Whilst it may seem that testability hinders TTM (why waste time writing tests when you can solely deliver executable code?), that's only part of the story. Introducing testability can certainly hinder the initial effort of the first release (testing may slow an initial delivery), yet it typically helps to make every subsequent delivery faster, and a return can be quickly recouped. This has great merit if we consider that 40%-80% of time is spent maintaining existing software.
ROI
Testability ensures quality, particularly as a regression tool. However, testing (and tests) is an investment in time and money, with a larger up-front (initial) cost, but lower overall costs overcome with every subsequent change (40%-80% Maintenance Mode).
Thus quality, automated tests (with a high degree of coverage), support fast turnaround, efficient working practices, Innovation, and nimbleness.
SELLABILITY
Testability improves the quality of software, in addition to (medium-to-long term) Productivity and Releasability. This allows cost-effective change to occur more rapidly, (potentially) increasing your perceived value to customers and thus, further supporting sales.
Most of this relates to Stakeholder Confidence. The sales team are confident that the product is of high quality (it's been robustly tested), whilst the customer perceives the quality (and gains confidence) by engaging with it, and the sales team.
TESTABILITY AT RUNTIME
Testability isn't solely about static software units, but also the ability to test these software services at runtime. Testability, then, should ensure software is testable in different contexts, and therefore using different forms of tests (e.g. acceptance, integration).
Testable (and releasable) software should make the provisioning environments a breeze; i.e. a secondary benefit is that we can swiftly and seamlessly provide prospective customers with a sandbox environment to play around with - thus further supporting the sales effort.
I've witnessed cases where testing is only practicable in a production environment (for payments, no less!), failing the testability quality, or where prospective customers aren't able to trial a product (arguably a form of testing) as aspects of it can't be easily simulated.
AGILITY
Testable software supports rapid change, innovation (through a testing Safety Net), and thus Agility. A team with the Safety Net of automated tests can be more radical, in both their thinking and in their solutions, betting bigger, and attempting things they wouldn't dare to do otherwise. Naturally, this dynamism leads to greater experimentation, learning, improvement, and business agility.
REPUTATION
Testability can also affect Reputation. Software that is difficult to test may:
- Be slow to deliver. An inordinate amount of effort is spent testing the solution functions (and continues to function) as expected.
- Require manual (testing) intervention and specialist skills.
- Be more expensive to test, or reduce our ROI. The cost must either be soaked up (eating into profits), or be passed on to the customer (affecting sales and existing relationships) - a form of waste.
- Isn't comprehensively tested. Software that is difficult to test requires either: (a) experts to support it (you must now consider project management and Capacity Planning), (b) requires additional time (and cost) to deliver, or (c) requires you to circumvent (Circumvention) aspects of testing to meet a deadline, thus affecting quality. None of these options are great.
TESTABILITY & BRANDING
The inability (or unwillingness) to sufficiently test a solution can have disastrous consequences. Recently, one well-known and trusted brand suffered catastrophic reputational damage when they migrated to a new system, in the main (it was reported), because they had failed to undertake sufficient testing before migrating their existing customers across. This is just one example of the catastrophic impact of insufficient testing, or poor testability.
The damages in these cases can be significant and severe, and include profit warnings, hundreds of millions of dollars in lost revenue, slashed share prices, and the departure of key personnel. All of this is reputational damage that can take years to recover from.
Finally, testability also promotes Stakeholder Confidence in our ability to (make) change. This is an important quality. By proving that a solution is robust and trustworthy, you are also proving your business and culture to be robust and trustworthy, which consequently, can increase sales, increase the morale of existing staff (retention), and promote your business to others as a good place to work.
SUMMARY
Important software services that don't support testability are a talisman to mediocrity. Testability promotes code quality and (test) automation, and automation promotes (amongst other things) product quality, Stakeholder Confidence, Productivity, Agility, Reputation, Innovation, and Sustainability.
The incentive to deliver untested code is appealing - we can probably deliver an initial release more quickly - but it is a false economy. The speed benefit gained from the initial release rapidly evaporates with each subsequent change to a solution with few automated tests. We also know that the later that we check for quality, the costlier a breakdown in quality is - both financially and in time [1]. Testability can counter this.
Finally, testability can also (indirectly) improve morale. Staff who employ a Safety Net are in a better position to “pioneer”, being more engaged and driven. Consequently, a culture of pioneering is likely to draw interest from both inside and outside of the business, leading to greater staff retention and a greater interest from prospective employees.
FURTHER CONSIDERATIONS