How to mock the java URL class with mockito

Unit testing classes that use java’s URL class

The java URL class java.net.URL models a remote resource location, and provides methods to retrieve resource content.  Common use cases are to retrieve resources over HTTP/S, but it can also be used to retrieve files or classpath resources.  Typically, the URL instance is constructed within your class method:

public byte[] readImage(String imageHref) {

    try {
        URL url = new URL(imageHref);
        InputStream inputStream = url.openStream();

        ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
        byte[] buffer = new byte[1024];

        int numRead;
        while ((numRead = inputStream.read(buffer)) != -1) {
            outputStream.write(buffer, 0, numRead);
        }

        outputStream.close();
        inputStream.close();
        
        return outputStream.toByteArray();
        
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}

To properly unit test this method, we are looking to test the happy path of a successful resource open and read, but also the path that falls into the catch block.

Problems unit testing this method

On the surface, this method looks pretty easy to test.  We can simply call it with a known good image href for the happy path test.  And call it with a non-existent image href to test the exception handling.  However, this approach has some problems and will make your test really brittle.

1. The test would be dependent on HTTP and a network connection.
   The test will be slower than it could be as it will be making real network requests, and the test will fail if the network connection is unavailable for some reason.
2. The tests are tied to resources out of our control.
   The tests will reference absolute URLs on a remote webserver. This tight coupling is bad. If the URL changes, the test will fail.  The test should be asserting a specific byte array is returned, so if the remote image resource is changed, it will return a different byte array, and the test will fail.
3. The test would be an integration test, not a unit test.

Possible solutions

There are a number of approaches to help address these problems, some of which are better than others.

1. Inject the constructed URL into the method.
   We could refactor the design such that the URL is constructed outside of the readImage method and injected in.  By doing this the test could provide a mock URL whose behaviour could be controlled.
   This has the advantage of making the test not dependant on network infrastructure, and also fixes the tight coupling of real resources.  The test is now a unit test again and will execute fast and consistently.
   Moving the construction of the URL into another class purely for test purposes might be considered a code smell.  Your overall design might benefit from this separation, but if the only reason you’ve created a URLFactory is to aid the unit testing of yourreadImage method, you might want to consider other options.
2. Make your test spin up a simple HTTP server, serving static resources from your test/resources folder.
   This approach brings the resources into your control, so fixes that part of the problem.  But the test is now definitely an integration test.  The test for a very simple method now has to run up a web server.  Even if you use a lightweight web server, this still feels wrong.
3. Make your test use file or classpath resources from your test/resources folder.
   Making the test use local resources such as classpath:your/package/image.gif certainly has some advantages.  The resources are in your control, and there is no dependency or requirement on external infrastructure.  Some purists might argue this is still an integration test.  And there is an argument that the test no longer reflects the runtime usage in that the test makes the method load a classpath resource, but at runtime the method is used to load a resource over HTTP.
4. Mock the URL class or supporting infrastructure.

Mocking URL with mockito

This post is all about mocking URL with mockito, so we will address option 4 above.

java.net.URL is a final class, so cannot be mocked with mockito. Even if it could be mocked, the current design of the method constructs the URL within the method, so it is not passed in as a dependency.  We should look at how the URL class works and re-think what components we are mocking.

The internals of URL

Internally, URL uses a static factory to create stream handlers for given protocols, and defines a static method setURLStreamHandlerFactory(URLStreamHandlerFactory fac) to set it.  The factory URLStreamHandlerFactory is an interface, which means it can be mocked.  Using the setter, we can inject a mock URLStreamHandlerFactory into URL

The URLStreamHandlerFactory interface defines one method URLStreamHandler createURLStreamHandler(String protocol)
URLStreamHandler is an abstract class, so we can define our own implementation and return it from our mock.

Putting it all together

First we define our URLStreamHandler implementation:

/**
 * {@link URLStreamHandler} that allows us to control the {@link URLConnection URLConnections} that are returned
 * by {@link URL URLs} in the code under test.
 */
public class HttpUrlStreamHandler extends URLStreamHandler {

    private Map<URL, URLConnection> connections = new HashMap();

    @Override
    protected URLConnection openConnection(URL url) throws IOException {
        return connections.get(url);
    }

    public void resetConnections() {
        connections = new HashMap();
    }

    public HttpUrlStreamHandler addConnection(URL url, URLConnection urlConnection) {
        connections.put(url, urlConnection);
        return this;
    }
}

Next, within the @BeforeClass method of our test class we can create our mock and inject it until URL. We can instantiate our UrlStreamHandler implementation, and instruct our mock to return it.

private static HttpUrlStreamHandler httpUrlStreamHandler;

@BeforeClass
public static void setupURLStreamHandlerFactory() {
    // Allows for mocking URL connections
    URLStreamHandlerFactory urlStreamHandlerFactory = mock(URLStreamHandlerFactory.class);
    URL.setURLStreamHandlerFactory(urlStreamHandlerFactory);

    httpUrlStreamHandler = new HttpUrlStreamHandler();
    given(urlStreamHandlerFactory.createURLStreamHandler("http")).willReturn(httpUrlStreamHandler);
}

Our HttpUrlStreamHandler contains a map of URLConnection that will be populated by each test.  Given that the HttpUrlStreamHandler is static, we need to reset this map before each test:

@Before
public void reset() {
    httpUrlStreamHandler.resetConnections();
}

Finally, our test methods can make use of this infrastructure:

@Test
public void shouldReadImage() throws Exception {
    // Given
    ImageLoader imageLoader = new ImageLoader();

    String href = "http://some.host.com/image.gif";

    URLConnection urlConnection = mock(URLConnection.class);
    httpUrlStreamHandler.addConnection(new URL(href), urlConnection);

    byte[] expectedImageBytes = toBytes(
            0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0x0A, 0x00, 0x0A, 0x00, 0x91, 0x00, 0x00, 0xFF, 0xFF, 0xFF,
            0xFF, 0x00, 0x00, 0x00, 0x00, 0xFF, 0x00, 0x00, 0x00, 0x21, 0xF9, 0x04, 0x00, 0x00, 0x00, 0x00,
            0x00, 0x2C, 0x00, 0x00, 0x00, 0x00, 0x0A, 0x00, 0x0A, 0x00, 0x00, 0x02, 0x16, 0x8C, 0x2D, 0x99,
            0x87, 0x2A, 0x1C, 0xDC, 0x33, 0xA0, 0x02, 0x75, 0xEC, 0x95, 0xFA, 0xA8, 0xDE, 0x60, 0x8C, 0x04,
            0x91, 0x4C, 0x01, 0x00, 0x3B);
    InputStream imageInputStream = new ByteArrayInputStream(expectedImageBytes);
    given(urlConnection.getInputStream()).willReturn(imageInputStream);

    // When
    byte[] imageBytes = imageLoader.readImage(href);

    // Then
    assertThat(imageBytes, equalTo(expectedImageBytes));
}

@Test
public void shouldFailToReadImageGivenInvalidHref() throws Exception {
    // Given
    ImageLoader imageLoader = new ImageLoader();

    String href = "http://some.host.com/bad-image-reference.gif";

    URLConnection urlConnection = mock(URLConnection.class);
    httpUrlStreamHandler.addConnection(new URL(href), urlConnection);

    IOException fileNotFoundException = new FileNotFoundException(href);
    given(urlConnection.getInputStream()).willThrow(fileNotFoundException);

    // When
    try {
        byte[] imageBytes = imageLoader.readImage(href);

        fail("Was expecting a RuntimeException with FileNotFoundException");
    }
    // Then
    catch (RuntimeException e) {
        assertThat(e.getCause(), equalTo(fileNotFoundException)); 
    }
}

private static byte[] toBytes(int... ints) {
    byte[] result = new byte[ints.length];
    for (int i = 0; i < ints.length; i++) {
        result[i] = (byte) ints[i];
    }
    return result;
}

References

• Original StackOverflow post that inspired this post