What if you got an unexpected result from an ActiveRecord query? If it’s important for the current page, rendering a 500 is a good idea. If the data is being used for a non-vital section, it’s best to render the page without the section:

@manager = Manager.find(params[:id]) rescue nil

<div id="sidebar">
  <% if @manager.nil? %>
    Data currently unavailable.
  <% else %>
    ...
  <% end %>
<div>

At Wealthfront, we don’t use ActiveRecord. Our frontends talk to backend services via RPCs (Remote Procedure Calls).

When an RPC fails unexpectedly, it raises a QueryException. This bubbles up to a 500 error page. If we’re grabbing data for non-vital fragments on a page, we mark the RPCs as failsafe:

class ManagersController
  def index
    @manager = service.get_manager(...)  # raises QueryException if it fails
    failsafe do
      # RPCs in a failsafe block will return nil if it fails
      @sidebar = service.get_sidebar_data(...)
      @footer = service.get_footer_data(...)
    end
  end
end

In our views, we make sure the data is available before using it:

<% if @sidebar.nil? %>
  Data currently unavailable.
<% else %>
  ...
<% end %>

 

Implementation

Clients need to know whether to raise an exception or return nil on errors. They’re initialized with the boolean failsafe.

class ServiceClient
  def initialize(failsafe)
    @failsafe = failsafe
  end

  def get_sidebar_data
    # make http request, raise a QueryException for status codes >= 400
  rescue Timeout::Error, QueryException => e
    if @failsafe
      Rails.logger.error("GetSidebarData failed")
      nil
    else
      raise
    end
  end
end

Our controller will keep track of the failsafe state and pass it to ServiceClient:

class ApplicationController
  private
  def failsafe
    @failsafe += 1
    yield
  ensure
    @failsafe -= 1
  end

  def service
    ServiceClient.new(@failsafe >= 1)
  end
end

Calls within failsafe can be inlined or batched together:

@result1 = failsafe { service1.call }
  
failsafe do  
  @result1 = service1.call
  @result2 = service2.call
end

 

Testing

It’s important for us to verify that pages render correctly with missing data. We’ve gone through all the trouble of not raising an exception when an RPC fails; we don’t want to raise any whiny nils in our views:

describe ManagersController do
  it "should render index" do
    service.stubs(:get_sidebar_data => ...)
    get :index
    response.status.should == 200
  end

  it "should still render index if failed RPCs occur" do
    stub_failsafe!
    get :index
    response.status.should == 200
  end
end

Controller specs are configured to render views. We use Mocha to stub out RPCs and return expected data. The method stub_failsafe! will stub all of our services to return nil when called within a failsafe block.

This will catch errors in templates like:

<%= @sidebar.something %>

That causes our example to fail with:

=> NoMethodError: undefined method `something' for nil:NilClass

since we should be checking that @sidebar is not nil before using it.