MARTYHAUGHT

I’m a big advocate of stubs and mocks in testing. In our current project we use Rspec which has mocking and stubbing as part of its framework. One of the big reasons that I like the use of stubs and mocks is in how it allows you to test in isolation. This allows you only to test the behavior of the class/method that you’re testing and not the rest of the application. This can very beneficial if your code uses an expensive resource (sockets/database) or something outside of your system (like a third party service). It can also keep you from having to set up a lot of dependencies in your own system if they are not directly used by your code. An added bonus is it can shorten your test execution time. But given my title there must be some aspect of this that makes me uncomfortable.

The first potential drawback to mocking (and stubbing) is the initial investment in complicated systems. It can take a bit more time than using the rest of your system. If you are testing a model that needs several associations to be established then you’ll need to mock each of the used associations and whatever related methods on those objects you might call. I consider this minor as the benefits of mocking outweigh the investment in most cases. Controller testing is the classic place where you will want to mock heavily. I have found that if the time to set up mocks gets too expensive I’ll skip it.

The second drawback is interface testing. I’ll use a simple example to illustrate this point.

class A
  def foo(bar)
    B.complex_calc(@baz, bar, :key1 => "something interesting", :key2 => 123)
  end
end

class B
  def self.complex_calc(param1, param2, options)
    result = private_method(param1)
    AnotherClass.something(result, param2, options)
  end
end

Usually in a spec I would mock out the complex_calc call when writing specs for class A. Such as:

it "should invoke the complex_calc method with baz, bar and the current options" do
  B.should_receive(:complex_calc).with(@baz, @bar, @options).and_return(123)
  @a.foo(@bar).should == 123
end

This certainly confirms that @a will make the call as you need it to. However, what happens if B’s method changes? You need to go find all the places that call it and change their mocks. Otherwise you now have specs that pass but your code is broken. Perhaps if you only have a few of these this isn’t a problem however if you use this technique often you’ll have a ton of these. Recently I’ve been burned by this where I changed the mocked interface and didn’t catch all callers of the changed method. So how would you deal with this? My current thought is to focus on mocking the sub layer. So in our example the spec would look like this:

it "should return 123 when given bar" do
  AnotherClass.stub!(:something).and_return(123)
  @a.foo(@bar).should == 123
end

In this case I left B alone so if the complex_calc method changes this spec has a better chance of catching it. I also used a stub over a mock as complex_calc already tests what happens internally and I only want to stub the objects it uses so I can keep it from hitting a DB or some other expensive resource.

The other thing that bothers me about mocking all interactions in your system is where do you test the integration? Hopefully you have integration/acceptance testing that does run the app through its paces without any mocking/stubbing. If all aspects of your code are tested ‘for real’ this way then I’m less concerned about liberal use of mocking interfaces. Otherwise, I’d fall back to the second layer approach outlined above.

The other factor that might come into play is the discipline of your team. If everyone is vigilant about interface changes and they know all uses and change the mocks then it seems fine to me. When your team adds new members will that vigilance continue or will you now find developers changing interfaces and breaking code when they got all specs to pass?

blog comments powered by Disqus