Kassis Netz / blog

Last time I was investigating how to find out which test or tests are dependent, making another test fail by just running before it. Boiled down to a small group of tests I am now digging into: why this happens.

I have found a command to run a small set of tests which will let my affected test fail:

$ bin/rspec --format documentation --seed 58021 spec/factories_spec.rb  spec/decorators/participation_decorator_spec.rb

Step 1: Reducing test scope

The factories_spec.rb is a spec that validates or factories created. I have a couple of factories listed and the given seed tests 2 of them before ParticipationDecorator is being tested and several others afterwards. With a simple if I excluded all but one, the evil one letting the test fail when it runs first, from being run at all. By that I have a neat test result with 2 simple specs, one guaranteed to run first while the second runs last and fails because of the first.

If I would have more specs inside participation_decorator_spec.rb I would also exclude everything except of the affected test – but currently I don’t.

Step 2: Realize The Actual Behavior

Or in other words: what exactly fails in the affected test?

That’s important to know when investigating because you have to know in which direction to look at.

Failures:

  1) ParticipationDecorator#winner_icon when participant wins returns the trophy icon
     Failure/Error: h.fa_icon("trophy") if object.winner

     NoMethodError:
       undefined method fa_icon for #<#<Class:0x007fcb13a4e670>:0x007fcb1385b390>

and the code

  def winner_icon
    h.fa_icon("trophy") if object.winner
  end

As I mentioned before there’s another spec failing together with this one. The problem there was that my h.link_to code inside the decorator suddenly includes the http://localhost:3000/ in local links which it hasn’t done before. Since I was checking the links, the test started to fail.

Let’s see how we can join these two problems.

Step 3: Investigate The Evil Spec

The first thing is quite easy: since it’s a factory spec it builds a factory object and validates it. I already reduced it to one special factory being tested, so I’m checking its attributes.

FactoryGirl.define do
  factory :post do
    user
    title  { generate :title }
    teaser { generate :text }
    body   { generate :text }
    ...
  end
end

It turns out that the :post factory has a user linked and commenting it will let my second test pass, so it’s obviously something to do with the :user factory or model.

When I started commenting out attributes of the :user factory to find out which one may be the problem I realized that every time the validation spec failed (because I removed a mandatory attribute), the affected test passed.

First I though I found the evil attribute, but unluckily this wasn’t the case. So it either has something to do with a validation that comes after those of the attributes I removed or with some after hook.

One interesting aspect with this is, that even when calling FactoryGirl.build(:post), the user reference will be created! That’s why I didn’t focus on the hooks at first.

For the test I could simplify my test scenario by * adding a let(:user) { create(:user) } on top of the affected test * run only this spec without dependency to some other

Having said that, I can let my tests pass by “simply” changing the factory to

FactoryGirl.define do
  factory :post do
    user { build(:user) }
    ...
    before(:create) do |post, evaluator|
      post.user.save
    end
  end
end

This ensures that a build is done on the referenced user when the post is built, and a create is performed with the user (by saving it) when the post is created. Any other ideas? I couldn’t find anything in Factory Girl.

However: this eliminates the symtoms for now, but it doesn’t get rid of the actual problem – and will come back as soon as a user object is needed to be created in combination with a decorator spec.

Step 4: Find The Root Cause

Starting with my reduced test set up, i. e. adding create(:user) to the decorator spec, I continue the path.

To cut the long story short here – after playing around with my user model line by line:

My user model uses devise with a lot of additions and removing :confirmable will let all specs pass. Adding it back again makes the tests with special helper functionality fail.

A quick look into devise reveals that it may send an email and a quick search on Google now brings this to light: Decorator is unable to access helper methods after sending an ActionMailer email..

The quick workaround to this is to add

FactoryFirl.define do
  factory :user do
    ...
    confirmed_at { DateTime.current }
  end
end

and hope that they’ll fix it soon – who knows what other problems exist.