Getting started with testability

At TestBash Netherlands, I said that, in my experience, a lot of testers don't really get testability. I would feel bad if I didn't follow that up with a starting point for expanding your mindset and explicitly thinking about testability day to day, and making your testing lives better! 

In large scale, high transaction systems testability really is critical, as compared to the vastness and variability of the world, testing done within organisations, before deployment, is limited by comparison. We need ways to see and learn from the systems we test where it matters, in Production.

Being able to observe, control and understand is central to testing effectively, and there are loads of resources and experience reports out there to help. I was/am inspired by James Bach, Seth Eliot, Matt Skelton, Sally Goble, Martin Fowler and a little bit of PerfBytes/Logchat, so lets see if it works for you! 

Overall Model:

Heuristics of Software Testability by James Bach

http://www.satisfice.com/tools/testable.pdf

In particular, the intrinsic testability of a system. I firmly believe that a system with high testability has high supportability in this regard. If you could build a system that gave those with operational responsibility with the ability to observe, control and understand what was happening, they would be stoked right? 

Videos:

Seth Eliot introduces testing in production with some compelling examples

https://vimeo.com/64786696

Sally Goble then talks about shifting testing away from current models and leveraging production:

https://vimeo.com/162635477

Articles:

Perfect Testing 

https://www.theguardian.com/info/developer-blog/2016/dec/04/perfect-software-the-enemy-of-rapid-deployment

Feature Toggles

https://martinfowler.com/articles/feature-toggles.html

Rethinking Pipelines

https://www.theguardian.com/info/developer-blog/2016/dec/20/testing-in-production-rethinking-the-conventional-deployment-pipeline

Testing Logging and Monitoring

https://www.infoq.com/articles/why-test-logging

http://www.slideshare.net/SkeltonThatcher/why-and-how-to-test-logging-devops-showcase-north-feb-2016-matthew-skelton

Data Driven Quality

http://www.setheliot.com/blog/2014/11/24/ddq1/

Signal vs Pass/Fail

http://www.ministryoftesting.com/2013/06/the-future-of-software-testing-part-two-testops-3/

Ask for Testability - Interesting Idea on Scriptable API's to drive the product

http://www.developsense.com/blog/2014/05/very-short-blog-posts-18-ask-for-testability/

A Map for Testability - Visualising testability in early product conversations

http://www.a-sisyphean-task.com/2014/07/a-map-for-testability.html

Putting Your Testability Socks On - Analysis of the SOCKS testability mnemonic

http://www.a-sisyphean-task.com/2012/07/putting-your-testability-socks-on.html

Audio:

Diagnostics

http://www.perfbytes.com/taxonomy/term/219

Logging

https://www.loggly.com/blog/business-justification-for-logging-logchat-podcast/

Exercises:

Architecture workshop from TestBash 2016

https://github.com/northern-tester/stratiform