Lessons in testing from software development
The more experience I get with design and engineering the more I realise how important testing is. In fact, I think it’s the most critical aspect of engineering.
The benefits of testing
When developing a new product or system it’s very difficult to have confidence in it without having tested it first.
Testing is used in every engineering discipline, but software engineers have developed good terminology for discussing it.
How software engineers approach testing
A software system is made up of blocks of logic which interact to create some desired output. The larger and more complex the system becomes the more likely it is that errors will creep in.
To combat this there are two main forms of testing used by software engineers - unit tests and integration tests. Unit tests are written for each block of logic, and test that a given input gives a desired output.
This leads to a large number of unit tests on any reasonably sized project (called a ‘test suite’). Each time a code change is made, this test suite can be run to make sure that all the components are still working as intended.
This gives the software engineer confidence that the changes they made haven’t broken the system.
The other type of testing is integration tests, where the overall interaction of the system is tested, rather than any individual component. An analogy is testing a car to make sure it handles properly.
The challenges with testing
Each time you write a test you are defining quite explicitly what the component of the system should do.
As more and more tests are written the system gets better defined. This is a good thing, because we want to arrive at a robust, well understood system.
However, this increasing rigidity makes it more expensive to make changes, because not only must the original code change, but also the tests written for it.
This is one of the reasons that tests are often written after most of the main code has been written. It formalises the function of the system.
It also make intuitive sense - you build something, then you test it. But there is an alternative approach, and one which is counterintuitive but very powerful - test driven programming.
Test driven programming
Rather than writing all the tests at the end, write the test before writing a new block of code.
This is powerful for three main reasons:
- It forces you to think carefully about what you want the code to achieve, and what the interface should be like (i.e. what variables it receives, and what it returns).
- It tends to stop you writing overly complex code. A block of code which does three things is harder to test than a block of code which just does one thing.
- The system is always fully defined in terms of tests.
The third point is especially important. These diagrams compare the two approaches:
In test-driven programming the system is always fully defined because the tests are written before the main code.
When testing afterwards the system is almost never fully defined.
Conclusions
Test driven programming has huge benefits over testing afterwards. But it can seem unnatural at first.
The terminology for software testing is also useful for other areas of engineering. When designing a new circuit board, unit tests can be written for each block of circuitry. For instance, this LED circuit should take an input of 20 mA and emit light intensity of 20 lumens. You then design the circuit to match the test.
Resources
- Kent Beck describes test driven development in his book Extreme Programming Explained.