What A Good Test...?

A colleague asked that question the other day. What is a good test? It is a really difficult question to answer. There are a number of properties that hold true for a good test. I will list some of them and discuss why I think they are important. Others may think that the order of importance is different or that other properties are more important then those that I list.

I will show some examples using Java and JUnit of what I think are bad tests and how they can be refactored to something better.

Tests are automated in my world. I always try to avoid manually testing. The main reason for this is that manual testing is slow and hard to repeat arbitrary many times. Therefore, test should be read as automated test in for the rest of this text.

Different types of tests

Tests come in many different shapes and forms. They have different purposes. Putting them in different categories is difficult and dangerous. A lot of people have opinions on the subject so it is like stepping in a mine field. Never the less, one way to categorise different tests may be this one (as suggested by Alexandru Bolboaca):

• Acceptance
• Integration
• System
• Unit

A good acceptance test is written in the domain language and clearly communicates the intent of a feature or story. The purpose is to have a way to measure when a feature can be considered done. It is formulated and understood by a domain expert rather than a developer struggling with the domain knowledge. It is expected to have a life time that is as long as the life time for product.

An integration test is usually slow. The purpose is to verify the integration between components that makes up the system. One such component could be a database.

A system test is usually very slow and could involve operations in GUI. The purpose is to verify that things that are expected to be possible to do actually is possible to do as a normal user. It could be a Selenium script that exercises the GUI through a browser by adding things to the database and then retrieve them.

A unit test must be blazing fast, running in milliseconds. The purpose is to develop the functionality incrementally and in a testable way. It is only using the core classes of the system and everything is executed in memory. This is where the use of tools like JUnit started and normally the result of test driven development, TDD. Unit tests will only test one thing by following the Single Responsible Principle, SRP.

Dividing test into these classes may differ from your opinion. Please suggest an alternative division if you think there may be a need for it.

One common thing for all categories above is that they follow the three R’s of automated tests:

• Rapid
• Reliable
• Relevant