Ruby on Rails Engines is one of the most powerful and underestimated features. It distinguishes Rails from other web frameworks.

What engines?

Engines are embeddable applications that can be integrated into your Ruby on Rails application. They are distributed as regular Ruby gems with models, controllers, and views.

The UI that comes with engines is incredibly useful, and most engines can be set up within 10 minutes, saving months of integration of microservices and external SaaS tools.

Here are the top eight Ruby on Rails engines:

1. Sidekiq - background jobs
2. Blazer - business intelligence
3. Lookbook - UI components previews
4. Flipper - Feature toggles
5. Split - A/B testing
6. PGHero - PostgreSQL monitoring
7. FastEntry - cache previewer
8. GraphiQL - GraphQL IDE

1) Sidekiq

Sidekiq is the de facto standard for background jobs in Ruby land. The engine provides a UI to monitor what is happening with your background jobs.

2) Blazer

Blazer allows you to write SQL queries and build charts and dashboards from them, making it an excellent tool for business intelligence.

But even if you don't use it as a business intelligence tool. It is a great developer tool. It can help you see what is in your database in local or staging environments.

3) Lookbook

I found this recently.

If you use ViewComponents to build UI elements instead of Action View Helpers, you should use it. It builds on top of previews and wraps them in Storybook like interface.

However, if you have ever tried to set up Storybook and React components, you have spent A LOT of time on it.

This gem takes about 10 minutes to set up, and the rest is just writing simple previews Ruby classes.

4) Flipper

Flipper provides infrastructure and UI for Feature toggle.

I have written before how to integrate with React - here.

Flipper has a paid version with features like permissions and audit history.

5) Split

Split provides infrastructure for A/B testing.

There are a lot of SaaS tools to do so; however, they are slowing your pages, cost a lot and take time integrate.

6) PGHero

It provides a performance dashboard PostgreSQL. It tells you things like - slow queries, unused indexes, table/index sizes, and more.

It comes from ? Instacard, which makes also Blazer and other wonderful open source projects.

7) FastEntry

Do you know what is in your cache? FastEntry shows you this. Simple and useful.

I have found a secondary use for it in development. I use the same Redis instance for cache, Sidekiq, and application data storage. It shows you everything in this Redis instance.

8) GraphiQL

Last but not least. GraphiQL is IDE for GraphQL. This engine provides quick setup and integration.

How to setup and secure engines?

I mentioned that engines are easy to set up. But talk is cheap. Let me show you the code. ?

Let's install Blazer, as it is one of the more complex ones to set up.

Step 1: Add to Gemfile

gem "blazer"

Step 2: Bundle install & database setup

bundle install

rails generate blazer:install
rails db:migrate

Step 3: Mount to the Rails application

# config/routes.rb
namespace :dev, constraints: Routes::AdminConstraint do
  mount Blazer::Engine, at: '/blazer'
  # ... other engines
end

Note: Routes::AdminConstraint ensures that this namespace is only available for certain users.

A simplified version can look something like:

module Routes::AdminConstraint
  extend self

  def matches?(request)
    return false if request.session[:user_id].blank?

    User.find(request.session[:user_id]).admin?
  end
end

In Production: Have database as ENV variable

ENV["BLAZER_DATABASE_URL"] = "postgres://user:password@hostname:5432/database"

That's it, you have a business intelligence tool ?

Conclusion

Engines have saved me a lot of time (and use of external services) over the years. I'm constantly on the lookup for new useful ones ?

If you have any questions or comments, you can ping me on Twitter or Mastodon.

Last year we very eventful for Product Hunt. On technical level we closed large system migrations like fully switch to Next.js for frontend, moved to TypeScript and rewrote our mobile app with React.Native.

Here is how our technical architecture looks currently:

I have given detail talks about each section of our systems.

• Backend - Rails, GraphQL, AWS, Docker, PostgreSQL (slides , video)
• Frontend - React, TypeScript, Next.js, Apollo (slides , video)
• Mobile - React Native, TypeScript, Apollo (slides, video)

Here is what a web request goes through to deliver a page from the site:

Overall, I'm very happy with our current system. It allows us to handle our traffic and ship fast and reliable.

Whats comes next?

Software is never done and this is true for us. We already had started improving on our foundations like:

• better core React components
• kinda, moving towards Domain Driven Design.
• improving on backend infrastructure for better performance and monitoring

For any questions or comments, you can ping me on Twitter.

If someone says, "Tell me about a technical failure you made", this is the story I'm going to share. ?

For the Product Hunt Collection project, I needed to implement a WYSIWYG HTML editor. The two choices at that time (March 2017) were Draft.js and Slate. I decided to go with Slate. It was easier to understand and had great examples that fit our needs.

Slate is a sound library and still serves us well to this day. The mistake I made was in the way I applied it to our project.

In the frontend, I wrapped and exposed Slate via:

• HTMLInput - React component that encapsulates the Slate HTML editor. Used in forms.
• slateToHTML - function which converts the Slate Editor state to a React component displaying its output HTML.

This encapsulation worked quite well.

<!-- HTMLInput in a form -->
<Form.Field name="description" control={HTMLInput} />

<!-- show HTML content -->
<p>{slateToHTML(collection.description)}</p>

Slate represents its state with a big nested JS object called the Editor. For the backend, I decided to store the Slate Editor state directly in a "JSONB" column.

My reasoning was the following:

• It will make the code simpler.
  • I only need to get the editor's state and pass it to the backend. Then, from the backend, put it back into Slate via slateToHTML.
• It will be more secure because we don't deal with raw HTML, and I can validate the JSON structure in the backend via GraphQL input types.
• We will have only 1~2 slateToHTML conversions per page, and there won't be a significant performance hit.

It took me a couple of days to build and ship a working HTML editor powered by Slate, but then...

What went wrong

The first bad sign was when we needed to send an email containing HTML generated by Slate. We needed to write a converter in Ruby that turned the Slate Editor state into an HTML string. Now, we had two places where we defined what our HTML Editor capabilities were. ?

Then Slate changed the structure of its Editor state and records in our database couldn't be directly passed to Slate. We used our Ruby converter and extended it to be a translation layer between the old and the new Slate Editor formats. We were doubling our work. ?‍♂️

There were many smaller annoyances like searching in HTML fields or validating based on HTML length. ?

We also needed different places to allow/disallow certain HTML and custom elements. We only validated this in the frontend, because validating Slate structure per field was tricky. ?

This was combined with an increased number of places that used Slate like Comments, Ship Messages, Goals, and others. We started hitting performance issues. ?‍♂️

Those issues started arising one by one in the span of a year. We were in boiling frog mode. Every fix was a workaround and made our system slower and more complex. ?

How we fixed this

We got stuck on an outdated Slate version - 0.19.30. It had a lot of bugs. The latest version at the time was 0.44.9 ?

During this time, we were working on Product Hunt Stories and needed to tune Slate even more. David was the person who took the challenge to fix our Slate issues.

The first step was to eliminate slateToHTML and just use sanitized HTML as string. The GraphQL API used the Ruby converter to return strings instead of Slate Editor objects.

David made HTMLInput to accept HTML as a string and convert it to Slate Editor state. With this, our frontend was thinking we deal only with strings, not JS objects.

It was a challenge to handle custom elements. There were a lot of those like the YouTube player, Tweets, gallery, and others. Those are now just simple HTML elements with data attributes:

<div data-component="tweet" data-tweet-id="1321095097883205632" />

Slate can parse those and convert them to React components like SlateTweetEmbed or SlateYouTubePlayer.

A special model concern was added to deal with HTML sanitization and validation:

class SomeModel < ApplicationRecord
  include SlateFieldOverride

  # "mode" defines which HTML tags are allowed
  slate_field :body, html_field: :body_html, mode: :everything
  
  # ...
end

The hardest thing was to migrate the database data. Fortunately, this went without any issues.

Our Ruby converter was battle-tested because it was already used to convert data for the frontend.

David migrated table by table, starting with the less relevant ones and eventually converting all of them.

The code for the migration looked something like:

Comment.not_migrated.find_each do |comment|
  comment.update!(
    body_html: Slate.to_html(comment.body),
  )
end

Having HTML to be stored just as string solved all issues we used to have:

• rendering HTML in React was fast
• display in emails
• allow/disallow certain tags per field
• easier search in HTML fields
• simpler data model

?

Lessons learned

• Don't depend on structures coming from places you don't control (like external libraries) - build a translation layer for those.
• Isolate external dependencies, so they only have one input and one output - if we didn't have the centralized HTMLInput and backend renderer, we would have had a lot of more issues.
• When adding a core component like a WYSIWYG Editor, which you expect to be widely used in your system - spend extra time to think through how its use is going to be expanded.
• Not being able to update a dependency sign that you haven't isolated this dependency enough.

Conclusion

I think our whole team and I learned a lot from this blunder. I hope it won't take us so much time to realize and course correct when we make such mistakes in the future.

People don't write tests because it takes too much time to write them. They lose a lot of mental energy on answering trivial questions like "Do we test this in our application?", "Should I test this?", "How do I test this?", "How do I set up my test", and so on.

For your team to write tests, your team members need two things: good conventions on how and what to test, and good tooling.

I spend a lot of time making the common use-cases easy to test. In Product Hunt, one such example is our Ruby GraphQL testing helpers.

We test three categories of GraphQL API code.

1. Helpers
2. Mutation classes
3. Resolver classes

Helpers

Those are helper module and classes like RecordLoader / RequestInfo / Context / RateLimiter and others. We don't have many of those. They are regular Ruby objects and are tested as such.

Mutations

Mutations in GraphQL are the operations that change your data. It is very, very important for those to be fully tested.

For each mutation, we test:

• Authentication and authorization rules
• Failure cases - validation, errors
• Happy paths

Mutations inherit from Mutations::BaseMutation class. I have written a blog post on the topic - here

Here is an example mutation:

module Mutations
  class QuestionCreate < BaseMutation
    argument :title, String, required: false

    authorize :create, Question

    returns Types::QuestionType

    def perform(attributes)
      question = Question.new(user: current_user)
      question.update(attributes)
      question
    end
  end
end

The way to test this, without extra tooling is:

describe Mutations::QuestionCreate do
  let(:user) { create :user }

  it 'creates a question' do
    context = Graph::Context.new(
      query: OpenStruct.new(schema: StacksSchema),
      values: context.merge(current_user: user),
      object: nil,
    )

    result = described_class.new(object: nil, context: context, field: nil).resolve(
      title: 'What email client do you use?',
    )

    expect(result[:errors]).to eq []
    expect(result[:node]).to have_attributes(
      id: be_present,
      user: user,
      title: 'What email client do you use?',
    )
  end
end

Yikes! ? Having to write something like this for every mutation will put off even the most enthusiastic developers.

The graphql-ruby classes aren't designed to be instantiated and run by you. The gem itself does this. Thus running them requires a lot of orchestration. That's why we have custom helpers to test mutations.

We have a custom helpers to test mutations.

Here is the updated example:

describe Mutations::QuestionCreate do
  let(:user) { create :user }

  it 'requires an user' do
    # executes the mutation with one-liner
    result = execute_mutation(
      title: 'test',
    )

    # verify that mutation returned this error
    expect_access_denied_error(result)
  end

  it 'creates a question' do
    # the one-liner, allow us to specify current user
    result = execute_mutation(
      current_user: user,
      title: 'What email client do you use?',
    )

    # "expect_node" verify that:
    # 1. there are no errors
    # 2. "node" is not null
    expect_node(result) do |node|
      expect(node).to have_attributes(
        id: be_present,
        user: user,
        title: 'What email client do you use?',
      )
    end
  end

  it 'requires a title' do
    result = execute_mutation(
      current_user: user,
      title: '',
    )

    # "expect_error" verify that:
    # 1. "node" is nil
    # 2. there is an error for given attribute and message
    expect_error result, :title, :blank
  end
end

This is a lot better. It clearly communicates what the business logic of this mutation is.

Here, you can find the gist of those helper methods.

Resolvers

Resolvers inherit from GraphQL::Schema::Resolver. They help us structure the code and make it easier to test. The alternative is to write the logic in the type classes. We tend to avoid this because classes get bloated and are harder to test.

The following is an example of a resolver that tells us if the logged-in user has liked a given object:

class Resolvers::IsLiked < Resolvers::Base
  type Boolean, null: false

  def resolve
    current_user = context.current_user
    return false if current_user.blank?
    
    Graph::IsLoaderPolymorphic.for(current_user.likes).load(object)
  end
end

Note: Graph::IsLoaderPolymorphic is batching helper. I have written about batching in GraphQL - here and here.

How will we test this?

We have two helpers for testing resolvers: execute_resolver and execute_batch_resolver.

describe Resolvers::IsLiked do
  let(:user) { create :user }
  let(:answer) { create :answer }

  def expect_call(object:, user:)
    expect(execute_batch_resolver(current_user: user, object: object))
  end

  it 'returns false when there is no user' do
    expect_call(object: answer, user: nil).to eq false
  end

  it 'returns false when the user has not liked the answer' do
    expect_call(object: answer, user: user).to eq false
  end

  it 'returns true when the user has liked the answer' do
    create: like, subject: answer, profile: user.profile

    expect_call(object: answer, user: user).to eq true
  end
end

Here, you can find the gist of those helper methods.

Conclusion

Test helpers can make testing more accessible. Our GraphQL helpers are just one example of this. People should think about the domain they are testing, not about test boilerplate.

If you have any questions or comments, you can ping me on Twitter.

Mutations in GraphQL ... mutate data. ?

At the time of this writing, Product Hunt codebase and related projects have 222 mutations. All of our mutations have the same structure. This enables us not only to have a consistent system but to build a lot of tooling around it to make our developer's life easier.

Here the Product Hunt guide on structuring GraphQL mutations:

Naming

There are only two hard things in Computer Science: cache invalidation and naming things
-- Phil Karlton

We have decided to name our mutations in the following pattern:

[module][object][action]

Here are some examples:

CommentCreate
CollectionPostAdd
PostVoteCreate
PostVoteDestroy
PostSubmissionCreate
ShipContactCreate
ShipContactDestroy

The reason to follow this approach is consistency and easier grouping. Autocompleting mutations names is even easier because you filter from module to object to the action.

Mutation shape

This is how a typical mutation looks like:

mutation CollectionPostAdd($input: CollectionPostAddInput!) {  
  response: collectionPostAdd(input: $input) {      
    node {
       id
       ...SomeFragment
    }    
    errors {    
      name  
      messages
    }
  }
}

It has the following elements:

• it is Relay compatible
• the result object is always named node
• it always has error array for user input validations
• in frontend we alias the mutation to “response.”

Let look at each one of those elementals:

Relay compatibility

We are using Apollo in our frontend. However, we try to have Relay comparable scheme, just in case this situation changes.

In order for a mutation to be Relay compatible, it is required to have a field named clientMutationId and accepted all of its arguments as inputs variable. inputs contains all arguments.

We tend to use clientMutationId when we don’t care about the mutation result. For example - TrackEventCreate .

inputs is more interesting. It is an input type, containing all values needed for the mutation.

It makes it very easy to work with Forms. Because we have a single argument object to hold the form state, we can just pass this object to the mutation. When we add/remove arguments from a given mutation, we don't have to change the mutation call at all.

I really like this concept.

Node field

We noticed that most mutations operate on a single object. Because Relay uses the term node a lot (example - connections). We decide to name the object returned from the mutation - node. This makes it easier for the frontend to handle it since it knows what to expect and where to look for it. Plus backend mutation can just return an object, making defining mutations in the backend easier.

In some rare occasions, we might need a second object to be returned from mutation. We support this.

Errors field

There are a lot of debates on how to handle mutation errors. Should the GraphQL error system be used, or should we return an array of error objects?

I prefer returning error objects. I split the errors into two buckets based on their cause:

1. those caused by system error
2. those caused by a user input error

In Product Hunt, we handle system errors with GraphQL system error, and we handle user input errors with error objects. Because of this, all our mutations have error fields, which return an array of Error objects:

type Error {
  name: String!
  messages: [String!]!
}

Our BaseMutation knows how to return it and our Form.Mutation knows how to map the errors to its input fields.

We use the name base for errors which can’t be attached to a particular field. Example: "you posted too many posts for today".

I have written about this before ? here.

Response alias

We alias the mutation field name to response, so we can simplify our frontend code:

const response = await mutation({ variables: $input });


responseNode(response)
responseErrors(resoonse)

Notice that this code applies to every mutation we have.

Backend tooling

BaseMutation

If you are working by schema first, enforcing those rules will be quite painful.

In Product Hunt, we use the revolver first approach to GraphQL development. We have a lot of built-in tools on top of GraphQL ruby gem.

One of those tools is BaseMutation. Every mutation in our system uses it.

Here is an example:

module Graph::Mutations
  class CollectionPostAdd < BaseMutation
    argument_record :collection, Collection, authorize: :edit
    argument_record :post, Post
    argument :description, String, required: false

    returns Graph::Types::CollectionPostType

    def perform(collection:, post:, description: nil)
      Collections.collect(
        current_user: current_user,
        collection: collection, 
        post: post, 
        description: description
      )
    end
  end
end

• it generates consistent mutations.
• it can fetch records
• it handles authorization
• it knows how to map returns to node field
• it knows how to handle raised errors from Ruby on Rails, validation, authorization and similar

The developer just has to think about:

• what is your input
• what are you returning
• what are the authorization rules
• implement

All the mechanics are handled by this class.

This is how we define mutations:

module Graph::Types
  class MutationType < Types::BaseObject
    # we have this small helper to reduce the noise when defining mutations
    def self.mutation_field(mutation)
      field mutation.name.demodulize.underscore, mutation: mutation
    end

    mutation_field Graph::Mutations::CommentCreate
    mutation_field Graph::Mutations::CollectionPostAdd
    mutation_field Graph::Mutations::PostVoteCreate
    mutation_field Graph::Mutations::PostVoteDestroy
    mutation_field Graph::Mutations::PostSubmissionCreate
    mutation_field Graph::Mutations::ShipContactCreate
    mutation_field Graph::Mutations::ShipContactDestroy

    # ...
  end
end

Frontend tooling

Having all wiring in the backend makes it very easy to build tooling for frontend. I already mentioned responseNode and responseErrors. We have them, but in reality, we use Form.Mutation and MuttationButton more.

Form.Mutation

I have written and spoken about this before. Here is a link to a post about it, and here is a link to a presentation about forms in general (here is a recording)

<Form.Mutation onSubmit={onComplete}>
  <Form.Field name="title" />
  <Form.Field name="email" control="email" />
  <Form.Field name="description" control="textarea" />
  <Form.Field name="length" control="select" options={LENGTH_OPTIONS} />
  <Form.Field name="level" control="radioGroup" options={LEVEL_OPTIONS} />
  <Form.Field name="speakers" control={SpeakersInput} />
  <Form.Submit />
</Form.Mutation>

This form handles:

• provides consistent UI for forms
• map fields and arguments
• knows how to pass inputs to mutation and interpret results
• protects against double submit
• handles validation errors and map them to fields
• handles the successful submission and passes node to onSubmit handler

MutationButton

The second main way mutations are triggered by buttons. We have a React component name MutationButton:

function LikeButton({ post, onLike }) {
  const mutation = post.isLiked ? DESTROY_MUTATION : CREATE_MUTATION;
  const optimistic = post.isLiked ? optimisticDestroy : optimisticCreate;

  return (
    <MutationButton
      requireLogin={true}
      mutation={mutation}
      input={{ postId: post.id }}
      optimisticResponse={optimistic(post)}
      onMutate={onLike}
      active={post.isLiked}
      icon={<Like Icon />
      label={post.likesCount}
    />
  );
}

This button handles:

• triggering the mutation with right arguments
• protects against double click
• handle optimistic updates
• handles clicks from not logged in users
• knows how to interpret the result of the mutations

Conclusion

Having all those structure and tooling around mutations helps the developers to focus on business logic and not worry about mechanics. This is a big win in my book.

In Part 1, I was showing how at Product Hunt we use GraphQL::Batch to solve N+1 when loading associations.

Those are the obvious first candidates for optimization in the GraphQL world.
But the N+1 queries is hidden in a lot of places.

For example, let's say we have the following query:

query {
  posts(date: 'today') {
    id
    name
    isVoted
  }
}

This isVoted returns whether the current user (viewer) have voted for this post.

The naive implementation would be something like:

class Graph::Types::PostType < GraphQL::Schema::Object
  field :id, ID, null: false
  # ..
  field :is_voted, function: Graph::Resolvers::IsVotedResolver.new
end

class Graph::Resolvers::IsVotedResolver < GraphQL::Function
  type !types.Boolean

  def call(post, _args, ctx)
    # handle not-logged in users
    return false unless ctx.current_user?

    # check if user have voted for a post
    ctx.current_user.votes.where(post_id: post.id).exists?
  end
end

This generates the following queries:

SELECT * FROM posts WHERE DATE(featured_at) = {date}
SELECT * FROM votes WHERE post_id={post.id} and user_id={user_id}
SELECT * FROM votes WHERE post_id={post.id} and user_id={user_id}
SELECT * FROM votes WHERE post_id={post.id} and user_id={user_id}
SELECT * FROM votes WHERE post_id={post.id} and user_id={user_id}
SELECT * FROM votes WHERE post_id={post.id} and user_id={user_id}
SELECT * FROM votes WHERE post_id={post.id} and user_id={user_id}
SELECT * FROM votes WHERE post_id={post.id} and user_id={user_id}
-- ...

Those are N+1. But there isn't a build in solution in Rails for those. Because of this, they are often unaddressed.

The solution we have in Product Hunt for this problem as of most of N+1 is to uses GraphQL::Batch:

class Graph::Resolvers::IsVotedResolver < GraphQL::Function
  type !types.Boolean

  def call(post, _args, ctx)
    # handle not-logged in users
    return false unless ctx.current_user?

    # `.for` ensures we get the same loader instance
    loader = VotesLoader.for(ctx.current_user)
    # `.load` adds the post id to the list of things to load
    loader.load(post.id)
    # return a future promise for this particular post load
    loader
  end

  class VotesLoader < GraphQL::Batch::Loader
    def initialize(user)
      @user = user
    end
    
    # this gets called after all `isVoted` were collected
    def perform(post_ids)
      # THE TRICK: load with one query only the voted post id from votes, not all posts
      voted_post_ids = @user.votes.where(post_id: post_ids).pluck(:post_id)

      # fulfill the future promises
      post_ids.each do |post_id|
        fulfill post_id, voted_post_ids.include?(post_id)
      end
    end
  end
end

This now generates only two queries.

SELECT * FROM posts WHERE DATE(featured_at) = {date}
SELECT post_id FROM votes WHERE post_id IN ({post_ids})

This problem exists with normal Rails applications. It isn't GraphQL specific. But I have rarely seen this being addressed. With GraphQL::Batch we can elegantly solve this problem.

When people see GraphQL their first question - what about N+1 problems.

This is a legit question. In Product Hunt as in every GraphQL powered project, we have the same issues.

For example, let's say we have the following query:

query {
  posts(date: 'today') {
    id
    name
    slug
    topics {
      id
      name
      slug
    }
  }
}

And the following type definition for a post:

class Graph::Types::PostType < GraphQL::Schema::Object
  field :id, ID, null: false
  field :name, String, null: false
  field :slug, String, null: false
  field :topics, [Graph::Types::TopicType], null: false
end

This would result the following queries:

SELECT * FROM posts WHERE DATE(featured_at) = {date}
SELECT * FROM topics JOIN post_topics WHERE post_id = {post_id}
SELECT * FROM topics JOIN post_topics WHERE post_id = {post_id}
SELECT * FROM topics JOIN post_topics WHERE post_id = {post_id}
SELECT * FROM topics JOIN post_topics WHERE post_id = {post_id}
SELECT * FROM topics JOIN post_topics WHERE post_id = {post_id}
SELECT * FROM topics JOIN post_topics WHERE post_id = {post_id}
-- ...

All those N+1s SQL queries ?

We don't only have additional queries but when a couple of posts have the same topics we load those multiple times.

The best solution, I have found to this problem so far was to use GraphQL::Batch gem from Shopify.

Adding the gem to your project is simple.

class Graph::Schema << GraphQL::Schema
  # ...

  use GraphQL::Batch

  # ...
end

The way batching works: Instead of executing the query immediately, it returns an instance of GraphQL::Batch::Loader. The same instance of a loader object is returned for a given field leaf. After all fields for a given GraphQL query leaf are collected, then the perform method of the loader is called with all requested records. preform then must load and return for each record the requested data.

Here is how are going to solve the N+1 for topics with GraphQL::Batch.

First, we change the post type to use a new resolver.

class Graph::Types::PostType < GraphQL::Schema::Object
  # ...

  field :topics, [Graph::Types::TopicType], null: false, function: Graph::Resolvers::Posts::TopicsResolver.new
end

Define the resolver itself:

class Graph::Resolvers::Posts::TopicsResolver < GraphQL::Function
  def call(post, _args, _ctx)
    # `.for` makes sure we return the same loader instance
    # so all leaves, so we can group data
    loader = TopicsLoader.for
    # adds a post to the list of posts to be loaded
    loader.load(post)
    # returns the loader, not the actual topics
    # this gets transformed into topics afterward
    loader
  end

  # Loaders represent promises and mechanism to 
  # postpone loading until we have all posts in the list
  class TopicsLoader < GraphQL::Batch::Loader
    # perform called with all the posts
    def perform(posts)
      # this is the built-in active record mechanism to 
      # preload associations into a group of records
      # association are loaded with the minimum amount of queries
      # if a couple of posts have same topics they would be loaded once
      ::ActiveRecord::Associations::Preloader.new.preload(posts, :topics)
      
      posts.each do |post|
        # returns topics for every post in the list
        fulfill post, post.topics
      end
    end
  end
end

Now we have a lot less queries ?

SELECT * FROM posts WHERE DATE(featured_at) = {date}
SELECT * FROM topics JOIN post_topics WHERE post_id IN ({post_ids})

Also, each topic is loaded only once ?

Since loading associations is a very typical problem, I have created a generic AssociationLoader. It can be found in this gist.

class Graph::Types::PostType < GraphQL::Schema::Object
  # ...

  field :topics, [Graph::Types::TopicType], null: false, function: Graph::Resolvers::AssociationResolver.new(:topics)
end

This weekend, I had the pleasure to attend GraphQL-Europe in Berlin, the first GraphQL conference in Europe.

Here is a recap of most notable things I learned there:

How GraphQL got from zero to wide adoption in one year

GraphQL spec was released less than an year ago. But it feels a lot more mature. This is because it was in production at Facebook for about 5 years before open-sourcing it.

The conference started with "GraphQL: Evolution or Revolution?" a great history lesson by Jonas Helfer.

In Five Years of Client GraphQL Infrastructure, Dan Schafer told us a couple of stories about how features like mutations were invented. There wasn’t much upfront design, they were just solving issue after issue. First in the frontend, then moving to the backend.

This connected very well with the Closing Keynote by Lee Byron. Where he talked about the future of GraphQL.

Those three talks convinced me that, GraphQL is not an accident.

Subscriptions

I haven't paid much attention of GraphQL Subscriptions in the past. they are now officially merged into the GraphQL spec.

In Realtime GraphQL from the Trenches, Taz Singh gave an interesting look into them.

I don't have many reasons to use them, mostly because Product Hunt doesn't need them at this time.

Persisted queries

Another feature I haven’t paid much attention to are persisted queries. Which is a miss on my part.

I had a couple conversations about persisted queries in between talks. Except for their obvious benefits - small requests, analytics and caching. I haven't realized they are also good from a security standpoint. Since if your GraphQL accepts only persisted queries in production, this makes impossible for somebody to open Chrome console and start firing expensive custom queries.

Handling errors

The Panel Discussion was very interesting. One of my major learnings from there - is that there isn't a standard way in GraphQL to handle form errors.

The format, I personally prefer looks something like this:

updateProfile(input: UpdateProfileInput) {  
  node: Profile
  errors: [Error]!
}

(I plan to write a blog post about form errors, someday)

I didn't know in GraphQL, you can return both a response and an error. It turns out that is a good strategy to handle failures in resolvers, which don't break the whole request. You can just return null for a given node and return errors explaining the failure.

{
  "errors": [
    {
      "message": "Service for fetching widget 1 failed",
      "locations": [ { "line": 4, "column": 5 } ],
      "path": [ "widget1" ]
    }
  ],
  "data": {
    "widget1": null,
    "widget2": {
      key: "value",
    }
  }
}

GraphQL in Ruby world

Ruby GraphQL gem, used by github and Shopify, is quite powerful and battle tested.

Netto Farah show a great solution for battling N+1 queries in Ruby project - GraphQL::QueryResolver.
Also in the same talk, he gave great tips about better tracking of GraphQL. If you are using GraphQL with Ruby definitely check his slides.

Launching GitHub's Public GraphQL API gave great tips about:

• handling authorization
• using GraphQL for backfilling legacy REST endpoints
• tips about schema design

Conclusion

Overall GraphQL-Europe was great. Kudos to Honeypot and Graphcool for organizing such great event.

p.s. I almost forgot! I finally found out what OData is :P.

When I started using GraphQL, I immediately saw, that SearchObject would be a perfect fit for search resolvers.

Having a GraphQL query to fetch the first 10 most recent published news posts would look something like this:

query {
  posts(first: 10, categoryName: "News", order: "RECENT", published: true) {    
    id  
    title  
    body  
    author {      
       id
       name
    }
  }
}

And it would have a corresponding SearchObject:

class Resolvers::PostSearch  
  include SearchObject.module  
  
  scope { Post.all }    
  
  option :categoryName, with: :apply_category_name_filter
  option :published, with: :apply_published_filter
  option :order, enum: %i(RECENT VIEWS LIKES)  
  
  # ... definitions of the option methods
end

So clean. ☀️

But then, PostSearch have to be connected with GraphQL Ruby gem:

PostOrderEnum = GraphQL::EnumType.define do
  name 'PostOrder'

  value 'RECENT'
  value 'VIEWS'
  value 'LIKES'
end  

Types::QueryType = GraphQL::ObjectType.define do
  name 'Query'

  field :posts, types[Types::PostType] do
    argument :categoryName, types.String  
    argument :published, types.Boolean  
    argument :order, PostOrderEnum
    resolve ->(_obj, args, _ctx) { Resolvers::PostSearch.results(filters: args.to_h) } 
  end
end

That isn't so bad. ?

But then, thinking about how can this code can change in the future:

• adding/removing options would involve going to both files
• adding a new order option would mean searching for the PostOrderEnum and manually sync it with PostSearch enum
• reusing PostSearch in other types, for queries like: query { user(id: 1) { posts(published: true) } }
  • requires copy and paste argument/type definitions
  • which makes updating the resolver even harder

Yikes! ? ?

This is where SearchObject::Plugin::GraphQL comes in. It puts type definitions and the resolver itself:

class Resolvers::PostSearch  
  # include the the plugin
  include SearchObject.module(:graphql)
    
  # documentation and type about this resolver
  # can be provided into the resolver itself
  type types[Types::PostType]
  description 'Lists posts'

  # enums or other types can also be nested
  OrderEnum = GraphQL::EnumType.define do
    name 'PostOrder'

    value 'RECENT'
    value 'VIEWS'
    value 'LIKES'
  end
    
  scope { Post.all }    
  
  # options just need to have a their type specifed.
  option :categoryName, type: types.String, with: :apply_category_name_filter
  option :published, type: types.Boolean, with: :apply_published_filter  
  # enums are automatically handled
  option :order, type: OrderEnum
  
  # ... definitions of the option methods
end

Then PostSearch can be used just as GraphQL::Function:

Types::QueryType = GraphQL::ObjectType.define do
  name 'Query'

  field :posts, function: Resolvers::PostSearch
end

Now, changing filter options requires changing only a single file. PostSearch can be reused in other types, by just adding function: Resolvers::PostSearch.

For more information check SearchObject::Plugin::GraphQL example.

During the Christmas break me and Mike were discussing a new feature at Product Hunt. The feature required scheduling an ActiveJobs when a user signs up, votes or submits a comment.

There is SignUp object which handles user registration. So scheduling a new background job there is quite simple:

module SignUp
  # ... handle user sign up

  def after_sign_up(user)
    WelcomeEmailWorker.perform_later(user)
    WelcomeTweetWorker.perform_later(user)
    SyncProfileImageWorker.perform_later(user)
    NewFancyFeatureWorker.perform_later(user) # <- new worker
  end
end

Unfortunately after_sign_up method was becoming quite large ☹️
Now imagine having to add NewFancyFeatureWorker to 10 other places ?

Those issues pushed us to create a simple wrapper around ActiveJobs, which we call KittyEvents.

Now in SignUp there is just one trigger for an "event":

module SignUp
  # ... handle user sign up

  def after_sign_up(user)
    ApplicationEvents.trigger(:user_signup, user)
  end
end

And there is a central place, where events are mapped to ActiveJobs Workers:

# config/initializers/application_events.rb
module ApplicationEvents
  extend KittyEvents

  event :user_signup, [
    WelcomeEmailWorker,
    WelcomeTweetWorker,
    SyncProfileImageWorker,
    NewFancyFeatureWorker, # <- new worker
  ]

  # ... other events
end

When an event is triggered, all ActiveJobs Workers for this events are scheduled and executed.

Another bonus, is when using KittyEvents, you only make a single Redis call to trigger any number of events. This shaves off precious milliseconds when using KittyEvents in request.

My last blog post - Dealing with form objects, got me thinking about form objects. I also had a 12 hours flight to San Francisco, where I was meeting my teammates at Product Hunt.

During that flight, I went back and checked all of my custom form implementations. I gathered them together in a folder and started combining and extracting the common parts. By the end of the flight I almost had what now is MiniForm. (I had to change its name several times since I'm not very good with names and most of the decent names are not available).

MiniForm allows the following code:

 class RegistrationForm  
  include ActiveModel::Model

  attr_reader :user, :account

  attr_accessor :first_name, :last_name, :email, :name, :plan, :terms_of_service

  # user validation
  validates :first_name, presence: true
  validates :last_name, presence: true
  validates :email, presence: true, email: true

  # account validation
  validates :account_name, presence: true

  # form custom validation
  validates :plan, inclusion: {in AccountPlan::VALUES}
  validates :terms_of_service, acceptance: true

  # ensure uniqueness
  validate :ensure_unique_user_email
  validate :ensure_unique_account_name

  def initialize
    @user    = User.new
    @account = Account.new owner: @user
  end

  def update(attributes)
    attributes.each do |name, value|
      public_send "#{name}=", value
    end

    if valid? & user.update(user_attributes) && account.update(account_attributes)
      account.users << user
      AccountWasCreated.perform_later(account)
    else
      false
    end
  end

  private

  def ensure_unique_user_email
    errors.add :email, 'already taken' if User.where(email: email).any?
  end

  def ensure_unique_account_name
    errors.add :name, 'already taken' if Account.where(name: name).any?
  end

  def user_attributes
    {first_name: first_name, last_name: last_name, email: email}
  end

  def account_attributes
    {plan: plan, name: name}
  end
end 

To become:

class RegistrationForm  
  include MiniForm::Model

  # `model` delegates attributes to models 
  # and copies the validations from them
  model :user, attributes: %i(first_name last_name email), save: true
  model :account, attributes: %i(name plan), save: true

  attributes :terms_of_service

  validates :plan, inclusion: {in AccountPlan::VALUES}
  validates :terms_of_service, acceptance: true

  def initialize
    @user    = User.new
    @account = Account.new owner: @user
  end

  # `update` calls `perform` if model is valid
  # and models are saved
  def perform
    account.users << user
    AccountWasCreated.perform_later(account)
  end
end

Today I released version 1.1 of my Search Object gem. It has two major new features.

Using instance method for straight dispatch

Suggested and developed by Genadi Samokovarov:

class ProductSearch
  include SearchObject.module

  scope { Product.all }

  option :date, with: :parse_dates

  private

  def parse_dates(scope, value)
    # some "magic" method to parse dates
  end
end

Classes mixed with SearchObject can be inherited

I had this setting on a branch for a long time. In one of the projects I worked on it would have been usefull to have a search base object:

class BaseSearch
  include SearchObject.module

  # ... options and configuration
end
 
# and used as 
class ProductSearch < BaseSearch
  scope { Product }
end

Its implementation actually might push me into an interesting refactoring.

Other minor features

• I added Rubocop for verifying the project
• I started testing against Ruby 2.2
• I stopped testing against Ruby 1.9

I don't use inheritance very often, but sometimes it is the best solution. Ruby doesn't care very much carrying when you overwrite a method. The method is just replaced by the new method. It doesn't care about arguments at all.

Since 2.0, Ruby support named arguments. I'm starting to use them more and more. But they are a bit tricky when overwriting.

Let's say we have a class which creates user objects named UserBuilder:

class UserBuilder
  def create(name: )
    puts "name: #{name}"
  end
end

When I have to create a CustomerBuilder, which creates customers (user with address), I can write it like:

class CustomerBuilder < UserBuilder
  def create(address:, name:)
    super name: name
    puts "address: #{address}"
  end
end

This works but creates a coupling between UserBuilder and CustomerBuilder. This coupling is a bit tricky to catch. Imagine the following scenarios:

• name become optional in UserBuilder
• name is renamed to full_name in UserBuilder
• name is split into first_name and last_name in UserBuilder

In all of those cases change in UserBuilder triggers changes in CustomerBuilder.

Such dependencies are toxic. Here is a better solution:

class CustomerBuilder < UserBuilder
  def create(address:, **args)
    super **args
    puts "address: #{address}"
  end
end

Now we can add, change, remove named arguments to UserBuilder#create and CustomerBuilder#create is not affected. Named arguments are just delegated down the change.