Our applications rarely work in isolation. Gone are the days where your app is just talking to a single database and storing files to local disc. Nowadays, apps talk to many external services like Sentry, Stripe, Twitter, etc.
External services are often added one by one in various places in a system, and it is easy to lose track of them. Because of this, at Product Hunt we have strict rules for dealing with external services. This blogs post shows and explains those rules.
Our main goals are to isolate those external services and to make it easy for team members to know why and how services are used.
1. Keep all services in a single place
All external services live in the
app/services/external folder and are namespaced under the
External module. All network-related code goes there. This includes calls to microservices we wrote ourselves.
In this way, we can see most of the outcoming network calls from our application just by opening this folder.
2. Wrap the external serice in a facade module
All external services are accessed via
External::[Service]Api facade module.
The following is our template for such a module.
# frozen_string_literal: true # Documentation # # - service: [url to service] # - manage: [url to manage tokens] # - api: [url to api documentation] # - gem: [gem we are using (optional)] # [- ... other links] module External::[Service]Api extend self # documentation: [documentation] def perform_action(args) # use HTTParty or gem for this external service end end
Usually, the API facades are just a bag of module methods.
Here is an example from our
# frozen_string_literal: true # Documentation # # - service: https://unsplash.com/ # - api: https://unsplash.com/documentation # - gem: https://github.com/unsplash/unsplash_rb # - portal: https://unsplash.com/oauth/applications module External::UnsplashApi extend self def search(query) Unsplash::Photo.search(query, 1, 12, 'landscape') end def track_download(id) photo = Unsplash::Photo.find(id) photo.track_download rescue Unsplash::NotFoundError nil end end
Even when the external service has a gem to access it, the gem is never used outside this module.
If the gem returns an instance of a response object, we try to wrap it with a class defined by us. We started wrapping those response objects quite late.
We expose only what we use from the services.
3. Storing credentials
Rails added support for encrypted credentials. Our newest applications use this mechanism for storing external services secrets.
We use the following commands to work with credentials.
EDITOR=vim rails credentials:edit --environment development EDITOR=vim rails credentials:edit --environment production
Credentials are stored in YAML. We always include a link to where a certain credential was taken because it is very irritating to search where token and secret was taken from—especially when dealing with Google APIs.
# Taken from: [url to where those credentials are taken] [sevice_name]_token: [...] [sevice_name]_secret: [...]
We are structuring our external services for the following reasons:
- Know precisely which services are external to the system
- Make it evident that we are interacting with an external system where network calls are involved
- Have an obvious surface area of external service, where we know what exact features we are using from them
- Having a facade makes it easier to add proper logging, caching, and error handling
- Simpler upgrade paths of APIs -- if an external gem has a breaking change, it only affects the facade object