Testers Guide to Myths of Unit Testing

One area that testers might be able to enhance their contributions to software development teams is how we perceive and contribute to unit testing. I believe testers busting their own illusions about this aspect of building something good would bring us much closer to developers, and help us realise what other layers of testing can cover most effectively.

Also, I want to do a talk about it, so I figured I would test the premise, see if potential audiences were into it. I put this on Twitter:

Working on a talk about what testers might believe about unit #testing & how we interact with developers creating unit tests. Any challenges/additions for my list below? #development #agile pic.twitter.com/4oT5HE4qs3

— Ash Winter (@northern_tester) December 19, 2017

30 replies with ideas tends to indicate that people might be into it. 

The List

I thought, as my final blog of 2017, I would provide a super useful list of the myths and legends we as testers might believe about unit testing:

• That developers always write unit tests
• That developers never write unit tests
• That testers can write unit tests for developers
• That developers know what unit testing is
• That testers know what unit testing is
• That a class is a unit to test
• That a function is a unit to test
• That two people mean the same thing when they say unit
• That anyone knows what a unit is in the context of their code
• That unit tests fill in the bottom of the test automation pyramid
• That unit tests remain in the bottom layer of the test automation pyramid
• That unit tests are inherently more valuable than other layers of tests
• That unit tests are fast to run
• That unit tests are always automated
• That lots of unit tests are undoubtedly a very good thing
• That unit tests can eradicated non determinism completely
• That unit tests are solitary rather than collaborative
• That test driven development is about testing
• That reading unit tests relay the intent of the code being written
• That unit test document the behaviours of code
• That when there are unit tests, refactoring happens
• That when there are no unit tests, refactoring happens
• That you never need to maintain and review unit test suites
• If it's not adding value through quick feedback it needs removing or changing.
• That unit tests sit outside a testing strategy for a product
• Because they exist, the unit tests are actually good
• Assertions are actually good. Checking for absence, as opposed to presence
• If you have a well designed suit of unit tests you don't need to do much other testing
• 100% code coverage for a given feature is evidence that the feature works as designed
• That code is always clean if it has unit tests
• Unit tests are about finding bugs
• That there is a unit to test
• That a failing test indicates what is wrong
• That one problem = 1 failed test
• That good unit tests are easy/hard (adapt based on your delivery) to write for non-deterministic functions
• "That unit test coverage is irrelevant to manual testing"? aka "Why look at them? They're JUST unit tests, we have to check that again anyways."
• That they may/may not believe that is a tester's responsibility to ensure code quality and consistency of the test suite (and that developers may believe the opposite)
• That unit tests don't count as "automation" if they do not use the UI
• That unit testing allows safe refactoring
• That the intent a developer has when they write the thing they call a unit test (guiding the design) is the same as the intent a tester has when they write the thing they call a unit test (discovery and confidence).
• That a large number of unit tests can replace integration tests.
• That unit tests evaluate the product.
• That false negatives ("gaps" or "escapes") in unit tests are a symptom of not having enough unit tests.
• Writing unit tests while developing the 'production' code is a waste of time, as the code will change and you'll have to rewrite them. 
• Having unit tests will prevent bugs
• That coverage stats give anything useful other than an indicator of a potential problem area.
• When and how often to run them. And how much confidence that actually gives you
• That code quality for tests doesn't matter as they're just tests
• When to write the unit tests (before/after the 'production' code 
• The difference between a unit test and and integration test
• That how much coverage you get with unit tests says anything about the quality of your test suite
• That you don't need additional tests because everything is unit tested
• That unit tests are the *only* documentation you need
• That they will be shared with the rest of the team
• TDD is a testing activity/TDD is a design activity/TDD is both/TDD is neither
• That the purpose of unit tests is to confirm a change didn't break something

The list is raw and some entries straddle disciplines as the world is big, fun muddle despite our efforts to compartmentalise. I hope its a useful guide to interactions with developers regarding this layer of testing. Next time a developer asks for an opinion on existing unit tests or help with writing new ones, have a look through this list and challenge your assumptions. After all, illusions about code is our business...

Thanks

Thanks to the following for their contributions*:

• Steven Burton
• Angie Jones
• Gav Winter
• James Sheasby Thomas
• Dan Billing
• Peter Russell
• Joe Stead
• Colin Ameigh
• Marc Muller
• Adrian McKensie
• Douglas Haskins
• Mat McLouglin
• Dan North
• Josh Gibbs
• Marit van Dijk
• Nicola Sedgewick
• Phil Harper
• Joep Schuurkes
• Danny Dainton
• Gwen Diagram

* If I forgot you, please tell me.