To Unit Test or Integration Test

To Unit Test or Integration Test … that is the question.

And that is the subject of this post. I will be exploring whether it is better to write Unit Tests or Integration Tests, or whether there is a sweet spot in the middle ground somewhere.

Striking the right balance

It’s hard to get the balance right. When written well, Unit Tests are small, easy to read, cover all logic paths, and run fast. But they don’t (or shouldn’t) span components across the stack.

By definition, Integration Tests test components wired up together – or integrated – across the stack. But they are harder to write. They require more setup, and more setup generally means more code. More code takes more time to write, and can be harder to read afterwards. And the tests themselves take longer to run, especially if the test setup needs to spin up contexts or other environments.

Enough with the theory, let’s cut to some code and see how we might test it.

Consider the following DAO class and it’s method to query a table for customers based on passed in data. The code uses Spring’s ‘NamedParameterJdbcTemplate’, but the concepts are transferable between different frameworks or languages.

public class CustomerDao {

    private static final String CUSTOMER_QUERY = "select * from customers";

    private final NamedParameterJdbcTemplate namedParameterJdbcTemplate;

    public CustomerDao(NamedParameterJdbcTemplate namedParameterJdbcTemplate) {
        this.namedParameterJdbcTemplate = namedParameterJdbcTemplate;
    }

    public List<CustomerDto> searchCustomers(String name, String email, String organisation) {
        String sqlQuery = CUSTOMER_QUERY;
        Map<String, Object> = queryParameters = new HashMap<String, Object>();

        /*
            code here to add the correct 'where' clauses to sqlQuery 
            and to populate the queryParameters map
        */

        return namedParameterJdbcTemplate.query(sqlQuery, queryParameters, new CustomerRowMapper());
    }
}

Obviously it’s not a complete class – just assume ‘CustomerDto’ and ‘CustomerRowMapper’ exist and do what they are supposed to. The key parts we need to look at are lines 15 thru 18. This is where the class would append ‘where’ clauses to the sql query and build the parameter map from the method arguments ‘name’, ’email’ and ‘organisation’. An implementation might look like:

if (name != null) {
    if (sqlQuery.contains(" where ") {
        sqlQuery = sqlQuery + " and lower(name) like :NAME";
    } else {
        sqlQuery = sqlQuery + " where lower(name) like :NAME";
    }
    namedParameterJdbcTemplate.put("NAME", "%" + name.toLowerCase() + "%");
}

// repeat for 'email' and 'organisation'

There are definitely better ways of doing this, but it should serve to demonstrate what we are trying to do. We are building a simple sql query with named placeholders and a corresponding map of named parameters, and that the query itself should operate on a case independent wildcard basis (assume Oracle syntax).

Let’s write the Unit Test

Unit Testing this should be straightforward, and we should be able to cover all logic paths using a Unit Test class such as the following. I am using jUnit and mockito, but as before, the concepts should be transferable to whatever tools you are using.

public class CustomerDaoTest {

    private NamedParameterJdbcTemplate namedParameterJdbcTemplate;

    private CustomerDao customerDao;

    @Before
    public void setup() {
        namedParameterJdbcTemplate = mock(NamedParameterJdbcTemplate.class);
        customerDao = new CustomerDao(namedParameterJdbcTemplate);
    }

    @Test
    public void shouldSearchCustomersGivenAllNullParameters() {
        // Given
        String name = null;
        String email = null;
        String organisation = null;

        String expectedSqlQuery = "select * from customers";
        Map<String, Object> expectedQueryParameters = Collections.EMPTY_MAP;

        // When
        List<CustomerDto> customers = customerDao.searchDao(name, email, organisation);

        // Then
        verify(namedParameterJdbcTemplate, times(1))
            .query(eq(expectedSqlQuery), eq(expectedQueryParameters), any(CustomerRowMapper.class));
    }

    @Test
    public void shouldSearchCustomersGivenJustName() {
        // Given
        String name = "A Person;
        String email = null;
        String organisation = null;

        String expectedSqlQuery = "select * from customers where name like :NAME";
        Map<String, Object> expectedQueryParameters = new HashMap<String, Object>();
        expectedQueryParameters.put("NAME", "%a person%");

        // When
        List<CustomerDto> customers = customerDao.searchDao(name, email, organisation);

        // Then
        verify(namedParameterJdbcTemplate, times(1))
            .query(eq(expectedSqlQuery), eq(expectedQueryParameters), any(CustomerRowMapper.class));
    }

    // repeat for other combinations
}

The above approach shows how we can provide complete Unit Test coverage of the ‘CustomerDao’ class covering all logic paths. We are testing just the DAO class and are mocking the external dependency. The tests are small, targeted, and will run fast.

Assuming the code to append to the sql string and add to the parameter map was factored out into a method rather than repeated for each parameter, then there are probably 4 combinations that will exercise all logic paths – all parameters null, the name parameter populated, the email parameter populated, and the organisation parameter populated. Maybe there’s an argument to also test combinations of populated parameters, but I wouldn’t test all possible combinations.

That’s awesome …. isn’t it?

For the most part, yes it is. As mentioned above, we can cover all logic paths with simple to write tests (that are also simple to read), and they run super fast. And we’ve not had to write an Integration Test with it’s associated bloat of spinning up a context and environment including a database (even an in-memory database such as H2).

So is there a case to test any more than this?

Do we need to do anything more? On the surface, the answer would be no – we’ve covered all logic paths in Unit Tests. But let’s dig a little deeper.

The class is building a string that ultimately is passed to the database driver. It contains database and environment specific terms. It contains the table name, and also potentially database vendor specific terms such as the % operators for wildcards, the keyword ‘like’, and the use of the ‘lower’ function. Though these are standard terms, the point I’m trying to make is that we could be using vendor specific operators.

Whilst our Unit Tests thoroughly test the sql string is constructed correctly, they do nothing to test the database driver or database itself will not error when presented with the query. What if the table name is not actually ‘customers’? What if the DBA’s upgrade the rdbms and it no longer supports the ‘like’ operator? (unlikely I know). Our Continuous Integration environment (you are running a CI stack aren’t you?) will not fail because all Unit Tests will continue to pass. And that’s the point of a CI stack – we want it to tell us of build problems as soon as possible.

So what do we do?

Integration Tests to the rescue

Yes, you guessed it, I’m going to propose an Integration Test to compliment our Unit Tests. Our Unit Tests test the logic paths within the class, and we should write one Integration Test to assert that the query is valid for the database runtime environment. A suitable test might look like the following:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = { "/test-app-context.xml" })
public class CustomerDaoIntegrationTest { 

    @Autowired
    private CustomerDao customerDao;

    @Test
    public void shouldSearchCustomersGivenAllParameters() {
        // Given
        String name = "bloggs";
        String email = "fred.blogs@somewhere.com";
        String organisation = "The Organisation Of Code Testing";

        CustomerDto fredBloggs = new CustomerDto("Fred Bloggs", "fred.blogs@somewhere.com", "The Organisation Of Code Testing");
        List<CustomerDto> expectedCustomers = Collections.singletonList(fredBloggs);

        // When
        List<CustomerDto> customers = customerDao.searchDao(name, email, organisation);

        // Then
        assertThat(customers, equalTo(expectedCustomers));
    }

} 

I’ve written this post from a Spring perspective, so I’m assuming the application has a Spring context where the ‘CustomerDao’ and JDBC datasource etc are all beans. As such, I’ve used the Spring annotations to spin up the context (defined in ‘test-app-context.xml’).

In our Unit Tests we new’ed up our ‘CustomerDao’ with a mocked ‘NamedParameterJdbcTemplate’. In contrast, the Integration Test uses ‘CustomerDao’ from the context (we @Autowire it in). The ‘CustomerDao’ will have its collaborators (‘NamedParameterJdbcTemplate’ in this case) provided by the context.

This post does not describe the detail of the context definition ‘test-app-context.xml’ as the post is discussing Unit vs. Integration testing, rather than how to define a Spring context.

How many tests?

I would only propose writing one Integration Test. We are using Unit Tests to assert the logic, and we need only one Integration Test to assert the query at runtime. There is no value in presenting all possible query combinations to the database driver. If the query with all ‘where’ clauses works, the other combinations will also work. I advocate using as many Unit Tests as necessary to test the logic and flow, and the minimal amount of Integration Tests to assert components (especially external dependencies such as databases) wired together.

The Sweet Spot

To summarise this post, I believe we have found the sweet spot. We can use Unit Tests to assert and cover all logic paths. When we mock external dependencies, we have small focussed tests that will run really fast. But Unit Tests alone can’t provide complete confidence in the system at runtime. To achieve that we can use carefully selected and well written Integration Tests. Whilst they typically require more code and setup, more resources of the CI stack, and take longer to run, if we are selective about how many we write and what they do they can compliment our Unit Tests to provide complete system confidence and early warning of problems in a runtime environment.