Blog of Rob Galanakis (@robgalanakis)

Linting pull requests

A couple weeks ago on Twitter, I joked about adding a way to bypass Cozy’s Pull Request linter by including #YOLO in the pull request description. It spawned an interesting discussion and a few people asked for more details about how the linter works.

We have some pretty thorough commit and pull request guidelines at Cozy, based on recognized best practices like these from Chris Beam. In other words, “update this” as a commit message is won’t do. After a few too many cut off commit messages and PRs missing links to the issues they were fixing, I finally got fed up enough to enforce these guidelines with a linting bot.

The entire linting project was just a few hours work*. When the web works, the productivity is incredible. When it doesn’t, you spend 3 hours debugging a JavaScript binding error. This was definitely a positive and productive experience.

Github has some excellent tutorials for building these sorts of bots. Check out Building a CI server. Your server gets some webhooks, and uses the Statuses API to update the PR. And, your bot can use the API to get whatever data you need. It doesn’t need a Git client or anything.

The jury is still out whether this is a good idea or not. I much prefer to have agreed standards that are continuously discussed and improved while being rigorously enforced, rather than have things happen haphazardly. There’s no room for poor quality to sneak in. Standards are clear. This strategy warrants its own post, but this post is just about the experience of building the linter.

I’ll dive into some example code, but that’s basically it. Like I said, simple**. The code below is very much based on the “Building a CI server” tutorial, so follow that if you need context.

First, set up a route to receive the webhook. This is mostly copy and paste from various tutorials:

      post '/github' do
        payload_body =
        return halt 500, "Signatures didn't match!" 
          unless webhook_verified?(payload_body)

        event = request.env['HTTP_X_GITHUB_EVENT']
        return unless ['pull_request', 'issue_comment'].include?( event )

        payload = JSON.parse(payload_body)
        return if payload['action'] == 'closed'

        if event == 'pull_request'
          repo = payload['pull_request']['base']['repo']['full_name']
          number = payload['pull_request']['number']
          repo = payload['repository']['full_name']
          number = payload['issue']['number']
        pr = @client.pull_request(repo, number.to_i)
        initialize_from_resource( pr ) # Sets some variables on this instance
        return 'Pull request processed!'

The process_pull_request sets the appropriate status based on problems with the PR. You are probably familiar with the statuses if you’ve used any Github PR bot.

        def process_pull_request
          @pending = create_status('pending', nil)
          if has_problems?
              'Issues with commit messages or missing context link. See details.')
              'PR looks good! (good commit messages, has issue/context link)')
          if @pending
              'PullRequestLinter errored while looking at this PR.')

        def create_status( status, description )
          org, rep = @repo.split('/')
            context: 'prlinter',
            description: description,
            target_url: "#{WebhooksApp.base_url(request: request)}/prlinter/#{org}/#{rep}/#{@number}")

I won’t delve into the has_problems? method, since it will just be based on your guidelines.
The only interesting thing about this setup is the target_url for the status.
If you navigate to it, you’ll get a breakdown of any problems with your PR.
We set up another route for that, and render out some markdown/HTML:

      get '/prlinter/:org/:repo/:pr_number' do
        initialize_from_params( params )
        ctx = context_problems
        commits = commit_message_problems
        md = [
          '### Pull Request Linter Report',
          "[#{@repo}##{@number}](#{@pr.html_url}) **#{@pr.title}**",
          "#{ctx.first ? '✖' : '✓'} Context: #{ctx.last}",
          "#{commits.first ? '✖' : '✓'} Commit Messages: #{commits.last}",
        if has_problems?
          md << 'Fix these problems before merging (if needed, you can force-refresh a PR Lint by commenting and deleting the comment).'
          md << 'If you need any help, see [Git Workflow](link to document).'
          md << "You're all ready to merge! See [Merging Your Branch](link to document) for what to do next."
        return render_md(md.join("\n"))

That gives you a report like this:

Screenshot 2016-03-21 23.19.02

That’s about it. I hope this helps you build your own Pull Request bots. Feel free to get in touch if you have any questions.

*: “a few hours” does not include deployment. Since this was part of an existing webhook app, deployment was not a problem.

**: So simple that, like any good CI service, it is missing tests…