Review of Endo-Testing: Unit Testing with Mock Objects

Endo-Testing: Unit Testing with Mock Objects is the paper written by Tim Mackinnon, Steve Freeman and Philip Craig where they introduced Mock Objects as a new technique for testing software.

I had this paper printed for quite some time but I haven't read it yet. After all the fuss about testing in the Rails community because of DHH's talk at RailsConf 2014 I decided to read the paper.

Below you will find what I highlighted along with my notes, comments and questions that I had while I was reading the paper. FYI, I wrote these notes around 2:00 AM and didn't review them after writing so things are about to get raw. I hope you enjoy them.

You will notice a common question that came up while I was reading the paper. The question is about dependency injection being used to make the testing process easier or if it is common to have DI in well designed systems, where its parts aren't highly coupled. I watched and read a couple of resources talking about DI and I couldn't find anyone saying that the main goal of DI is to make testing easier.

You might be thinking that I was silly just by having this doubt but the countless examples of straw men that appear these days can lead people to have such thoughts. :(

Reading the paper was a good exercise to know how they introduced the concept, the ideas, motivations and also to realize that the paper is still relevant nowadays.

Introduction

We propose a technique called Mock Objects in which we replace domain code with dummy implementations that emulate real code. These Mock Objects are passed to the target domain code which they test from inside, hence the term Endo-Testing.

  • This is the definition of the terms. One question that comes up from this is about the statement that says that the mocks are passed to the target domain code. Could this lead to code that is created to make it easier to test or is this a natural process (injecting the dependencies) to make the code less coupled?

…we use our tests and stubs to drive the development of our production code.

Unit tests written with Mock Objects have a regular format that gives the development team a common vocabulary.

  • A specific style for the tests?

Unit testing with Mock Objects

An essential aspect of unit testing is to test one feature at time;

  • Kinda easy to spot if you know the meaning of the word “unit”.
  • How others define unit testing?
    • Martin Fowler wrote a good post about it1.

Test code should communicate its intent as simply and clearly as possible. This can be difficult if a test has to set up domain state or the domain causes side effects.

  • This is one of the advantages of doing TDD because soon enough you can see that things are difficult to setup so it might indicate that your code is too complex and is a good candidate for refactoring into less complex code. Remember that this is not a rule and fuck straw men.

    • I’m not saying that if you write your tests after writing production code you won’t be able to identify that the code is complex. I just think that you might save time because you chose to write the tests first.

A Mock Object is a substitute implementation to emulate or instrument other domain code. It should be simpler than the real code, not duplicate its implementation, and allow you to set up private state to aid in testing. The emphasis in mock implementations is on absolute simplicity, rather than completeness.

  • Could the simplicity lead to brittle tests?

    • As with everything else in life, it could. One might say that is too smart to avoid the brittleness but we know that is not always the case and people make mistakes. Although I do not think that the majority of the cases will be brittle tests. As the developer gains experience he will learn and improve the tests so they become more robust.
  • Completeness can be achieved by using a combination of types of tests and the authors later say that unit testing with Mock Objects don’t remove the necessity to write functional tests.

Not just stubs

Mock Objects that become too complex to manage suggest that their domain clients are candidates for refactoring, and we avoid chaining Mock Objects.

  • This is basically the same thing that is said about having tests that are too difficult to setup.

…our coding style of passing stub objects as parameters, rather than relinking the domain code, clarifies the scope of unit testing, and reduces the risk of mistakes during a build.

  • Dependency injection again. The tests are the reason or they just take advantage of DI being a technique to reduce coupling?

  • Not relinking the domain code makes the tests more robust since they do not need to know much about the inner aspects of the domain code. You work with the behavior, the interface of the domain code, not how it is exactly implemented.

Why use Mock Objects?

Localising unit tests

Deferring Infrastructure Choices

…we might with to write functionality without committing to a particular database. Until a choice is made, we can write a mock class that provides the minimum behaviour that we would expect from our database.

  • This is a huge controversial point, specially in the Rails world where many will say that you probably never change your database, or the ActiveRecord library makes it transparent. Testimonials of real world cases where similar changes happened would be welcome.

  • I remember that the authors of GOOS2 (Freeman is one of them) advocate something similar in their example of bidding system.

  • This reminds me of the hexagonal architecture.

  • Forget the big change, like changing the database and imagine a situation where you don’t have to think about the real component that is going to be used. You can write the tests and start defining an API for such component that is still undecided. This API/interface can then help in your choice of component.

The mock code also gives us an initial definition of the functionality we will require from the database.

  • See my previous note above.

Coping with scale

Unit tests, as distinct from functional tests, should exercise a single piece of functionality. A unit test that depends on complex system state can be difficult to set up, especially as the rest of the system develops.

  • Don’t even get me started about straw men on DHH’s posts. I highly recommend reading Gary Bernhardt’s post3 about this.

  • By testing a single piece of functionality we can focus in one task, reducing complexity and this is one thing that I can’t see being bad. If we change our way to focus more on functional tests, as DHH proposes, we are going to lose some of this focus since functional tests generally tend to touch several parts of the real code. Yeah yeah, I know that you can assert just one expected behavior in a functional test but chances are that testing the creation of a user (in the context of a web app) will touch more code by filling in forms, passing through the controller and then saving the new user to the database. Contrast that by having a test that directly calls a single method.

  • Another thing that comes up in my mind is about Functional Programming. In FP you will usually have a function that does one specific task so when you test it you will not be dealing with several different parts of your code reducing the chances of being lost about where or why some failure occurred in your test.

    • Could this be considered a reason to shift towards FP? I’m not serious here. ;)

No stone unturned

The test has no dependencies on components outside the development system and is insulated from other possible real world failures.

  • This refers to using mock objects instead of real components, like an external server or service.

This style of test is repeated for other types of failure, and the entire test suite documents the possible server failures that our client code can handle.

  • I don’t believe that tests should stand in for documentation and I’m not saying that the authors proposed this. But we can’t discard the value of tests. Tests are one type of documentation and is always a good thing to have more documentation.

Better tests

Failures fail fast

…a mock implementation can test assertions each time it interacts with domain code and so is more likely to fail at the right time and generate a useful message.

  • ‘nuff said.

Refactored assertions

…assertions are built into Mock Objects and so are applied by default whenever the object is used.

  • This refers when you have a mock class that is used by several tests.

During development, the authors have come across situations where assertions in their Mock Objects have failed unexpectedly. Usually this is a timely warning about a constraint that the programmers have forgotten, but sometimes this is because the failing constraints are not always relevant. These cases suggest candidates for refactoring of either the domain code or Mock Objects, and help to push the developers towards a better understanding of the system.

  • No rules, laws, nothing. Just suggestions, YMMV.

Effects on coding style

Developing with Mock Objects reduces the need to expose the structure of the domain code. A test knows more about the behavior and less about the structure of tested code.

  • Already pointed before that the focus shifts towards behavior which is also mentioned several times in the GOOS book.

Developing with Mock Objects encourages a coding style where objects are passed into the code that needs them. This makes substitution possible and reduces the risk of unexpected side-effects.

  • The question persists, what is the reason: easier to test or less coupling?

  • DHH’s disciples believe that you will never be required to change your database or another big component in your system. And they also don’t like laws (hello Demeter) because generally people tend to use them 100% of the time and that leads to undesired outcomes. The funny thing here is that this is a contradiction since you cannot say that 100% of the projects out there will never change a big component so you must always think that what might not suit you could suit another person and that does not make you or him wrong.

  • Another thing that came to my mind is that somehow passing other objects as arguments kinda resemble what is done in the functional land where a function depends on its inputs. This can be easily related to the fact that in functional land you generally tend to have less coupling.

…developing with Mock Objects tends to push behaviour towards Visitor-like objects that are passed around…

  • See why I keep saying that the big question persists? It’s everywhere.

Interface discovery

When writing code that depends on other related objects, we have found that developing with Mock Objects is a good technique for discovering the interface to those other objects.

  • This is one of the biggest selling points of TDD. My opinion is that TDD helps me with interface discovery but it might not work that way for everyone.

Limitations of Mock Objects

…unit testing will not catch failures that arise from interactions between components.

This is why functional tests are still necessary, even with good unit tests.

In some cases it can be hard to create Mock Objects to represent types in a complex external library.

  • What is the best thing to do in such cases?

This process can be costly and sometimes must be weighed against the benefits of having the unit tests. However, when only a small part of a library needs to be stubbed out, Mock Objects is a useful technique for doing so.

  • Did someone already pointed that “YMMV” is a thing?

A Pattern for unit testing

  • The authors show the common format that they identified for their unit tests.

    • Create instances of Mock Objects.
    • Set state in the Mock Objects.
    • Set expectations in the Mock Objects.
    • Invoke domain code with Mock Objects as parameters.
    • Verify consistency in the Mock Objects.
  • Remind that the paper was written with Java examples so this format may be simplified in your language/testing framework of choice.

With this style, the test makes clear what the domain code is expecting from its environment, in effect documenting its preconditions, postconditions, and intended use.

Notes

  1. http://martinfowler.com/bliki/UnitTest.html 

  2. http://www.growing-object-oriented-software.com/ 

  3. https://www.destroyallsoftware.com/blog/2014/tdd-straw-men-and-rhetoric