Organizing External Services in Rails

Our applications rarely work in isolation. Gone are the days where your app is just talking to a single database and storing files to local disc. Nowadays, apps talk to many external services like Sentry, Stripe, Twitter, etc.

External services are often added one by one in various places in a system, and it is easy to lose track of them. Because of this, at Product Hunt we have strict rules for dealing with external services. This blogs post shows and explains those rules.

Our main goals are to isolate those external services and to make it easy for team members to know why and how services are used.

1. Keep all services in a single place

All external services live in the app/services/external folder and are namespaced under the External module. All network-related code goes there. This includes calls to microservices we wrote ourselves.

In this way, we can see most of the outcoming network calls from our application just by opening this folder.

2. Wrap the external services in a facade module

All external services are accessed via External::[Service]Api facade module.

The following is our template for such a module.

# frozen_string_literal: true

# Documentation
#
# - service: [url to service]
# - manage: [url to manage tokens]
# - api: [url to api documentation]
# - gem: [gem we are using (optional)]
# [- ... other links]

module External::[Service]Api
  extend self

  # documentation: [documentation]
  def perform_action(args)
    # use HTTParty or gem for this external service
  end
end

Usually, the API facades are just a bag of module methods.

Here is an example from our External::UnsplashApi:

# frozen_string_literal: true

# Documentation
#
# - service: https://unsplash.com/
# - api: https://unsplash.com/documentation
# - gem: https://github.com/unsplash/unsplash_rb
# - portal: https://unsplash.com/oauth/applications

module External::UnsplashApi
  extend self

  def search(query)
    Unsplash::Photo.search(query, 1, 12, 'landscape')
  end

  def track_download(id)
    photo = Unsplash::Photo.find(id)
    photo.track_download
  rescue Unsplash::NotFoundError
    nil
  end
end

Even when the external service has a gem to access it, the gem is never used outside this module.

If the gem returns an instance of a response object, we try to wrap it with a class defined by us. We started wrapping those response objects quite late.

We expose only what we use from the services.

3. Storing credentials

Rails added support for encrypted credentials. Our newest applications use this mechanism for storing external services secrets.

We use the following commands to work with credentials.

EDITOR=vim rails credentials:edit --environment development
EDITOR=vim rails credentials:edit --environment production

Credentials are stored in YAML. We always include a link to where a certain credential was taken because it is very irritating to search where token and secret was taken from—especially when dealing with Google APIs.

# Taken from: [url to where those credentials are taken]
[sevice_name]_token: [...]
[sevice_name]_secret: [...]

Conclusion

We are structuring our external services for the following reasons:

• Know precisely which services are external to the system
• Make it evident that we are interacting with an external system where network calls are involved
• Have an obvious surface area of external service, where we know what exact features we are using from them
• Having a facade makes it easier to add proper logging, caching, and error handling
• Simpler upgrade paths of APIs -- if an external gem has a breaking change, it only affects the facade object