Build a CI/CD pipeline for your Jekyll site

Since setting up my Jekyll site, I thought that being disciplined enough to continue writing content regularly would be the main blog-related problem I would be dealing with.

That is, until I ran HTMLProofer over the site, and it spat out a bunch of validation issues that showed me I had a significant amount of problems including:

Some of the links I had in older posts were returning 404 messages, even though the links had worked when I first wrote the posts…
The sitemap generated by jekyll-sitemap apparently contained invalid jekyll-archives-generated archive page links (eg https://www.paulfioravanti.com/tags/jekyll/), which I thought was strange as they all worked in my development environment…

It turns out that:

I had references to code on Github that referred directly to the master branch of a codebase, rather than use a permalink, which would refer to a file at the time I accessed it (always use permalinks when linking to code on Github to avoid this headache)
The list of supported Jekyll plugins on Github Pages does not include jekyll-archives (I forgot/didn’t check), so although the archive links worked in my local development environment, when they were added to the locally-generated sitemap, and then an attempt made to check their production-side links, they all 404-ed.

The issues that HTMLProofer brought up raised some questions around the quality of the site code and post content:

Links in posts could become stale, and I would never know about it, since I do not ever go and manually re-check all the links in every post
I had no process that could stop me from inadvertently deploying bad code, or deploying plugin code that would just be ignored by Github Pages, leading to production environments not matching what I was seeing in development
Limitations on Github Pages plugin availability mean I could also not take advantage of some plugins that could help with search engine optimisation (SEO), like those that minify your HTML/CSS/JS files

I still want to keep Github Pages as the deployment platform for my blog (at least, for now), but it just seems really limiting, so what options do I have? Well, thanks to Derek Smart and his post Supercharge GitHub Pages with Jekyll and Travis CI, I learned that I should:

Break free of the constricting GitHub Pages Ruby Gem
Replace it with the Jekyll gem, and a list of any Jekyll plugin gems I please
Create a build pipeline using Travis CI.
Have Travis test my code, build the site with all the plugins I want, and then deploy the built site directly to the master branch of my Jekyll Github repository

Let’s see about getting this done!

Fancycrave Image — Photo originally by Fancycrave on Unsplash

Switch to Jekyll gem

My Gemfile initially looked similar to the Github specification for Jekyll sites that get published to Github Pages:

Gemfile

source "https://rubygems.org"
ruby "2.5.3"

group :jekyll_plugins do
  gem "github-pages", "192"
  gem "jekyll-archives", "~> 2.1"
  gem "jekyll-include-cache", "~> 0.1"
  gem "jekyll-remote-theme", "~> 0.3"
end

After migrating over to the Jekyll gem, it changed over to be something like:

Gemfile

source "https://rubygems.org"
ruby "2.5.3"

gem "jekyll", "~> 3.8"

group :jekyll_plugins do
  gem "jekyll-archives", "~> 2.1"
  gem "jekyll-include-cache", "~> 0.1"
  gem "jekyll-remote-theme", "~> 0.3"
  gem "jekyll-feed", "~> 0.11"
  gem "jekyll-gist", "~> 1.5"
  # and a bunch of other jekyll plugin gems...
end

In order to determine what other gem plugins I should include under the :jekyll_plugins group, now that the github-pages gem was not bringing them in for me, I referenced the Github Pages gem dependencies, as well as the plugins key in my site’s _config.yml file, and then manually added any that I knew I was using.

For reference, here is my blog’s Gemfile at the time of this writing.

Set up Travis to test and deploy site

The Supercharge GitHub Pages with Jekyll and Travis CI article perfectly describes, with appropriate detail, how to set up Github and Travis to build and deploy a Jekyll site, so I will only repeat a summary here with links to Github and Travis documentation:

Since Github Pages publishes code pushed into the master branch, create a new release branch on your repository and set it to be the default branch. This effectively makes release the new master branch for development purposes, and master will, moving forward, only ever hold the contents of the built Jekyll site
Activate your Jekyll site on Travis
Create a personal access token for use with the Jekyll site
Copy your new personal access token into an environment variable in the Travis repository settings for your Jekyll site (suggested name: GITHUB_TOKEN)
Create a .travis.yml configuration file in the root of your Jekyll site to tell Travis what to do

Build Stages

What Travis should do specifically is run all the lints and tests for the Jekyll site. Then, if everything passes, Travis should build the site, and deploy it to the master branch of the site Github repository, at which point it gets automatically published to Github Pages.

Travis Build Stages enable us to do exactly that, so let’s see how to do that in configuration:

.travis.yml

sudo: false
dist: trusty
language: ruby
cache: bundler
rvm:
  - 2.5.3
bundler_args: --without development
env:
  global:
    - NOKOGIRI_USE_SYSTEM_LIBRARIES=true
addons:
  apt:
    packages:
      - libcurl4-openssl-dev
branches:
  only:
    - release
jobs:
  include:
    - stage: Test
      before_script:
        - npm install -g sass-lint htmllint-cli markdownlint-cli
      script:
        - JEKYLL_ENV=production bundle exec jekyll build
        - htmllint _includes/stripped_markdown.html
        - markdownlint _posts _drafts _pages README.md
        - sass-lint --verbose --no-exit
        - bundle exec htmlproofer _site --allow-hash-href --assume-extension --url-ignore "/localhost/" --http-status-ignore "999"
    - stage: Github Release
      script:
        - JEKYLL_ENV=production bundle exec jekyll build
      deploy:
        provider: pages
        local-dir: ./_site
        target-branch: master
        name: Travis Deployment Bot
        skip-cleanup: true
        github-token: $GITHUB_TOKEN
        keep-history: true
        on:
          branch: release

If you have used Travis to build a Ruby project before, you will no doubt recognise some of the configuration entries listed here, so I will outline some notes only about the potentially unfamiliar parts of this file, and why they’re there:

The NOKOGIRI_USE_SYSTEM_LIBRARIES=true statement and use of the libcurl4-openssl-dev library are for using HTMLProofer on Travis: the former speeds up installation of HTMLProofer, and the latter is so checks on external links referenced in posts can actually be performed
We only ever want to test and deploy the release branch, so make sure we ignore pushes to any other branch
Build stages are defined under the jobs key, and here we have two: Test and Github Release
All the lint checks are described in my previous blog post Setting up a Jekyll Blog. Check it out for more information on what they are linting, so you can get a better idea of whether you should be changing any parameters to suit your own site’s needs
The htmlproofer command line interface (CLI) application has lots of options, but the reasons I use certain options are the following:
- --allow-hash-href - The build will fail on the first and last post entries if this isn’t allowed because I have pagination “previous” and “next” buttons that have href="#", and hence considered ‘links to nowhere’
- --assume-extension - Jekyll 3 supports extension-less permalinks, and my blog uses them, so this flag needs to be here to prevent errors
- --url-ignore "/localhost/" - I have tutorial posts that have explicit link references to localhost, so I don’t want them considered “proper” external links that need to be validated
- --http-status-ignore "999" - LinkedIn doesn’t seem to like crawlers, and hence sends back 999 errors, even if a link to them is valid
The configuration under the deploy key is adapted from a Travis example configuration of deploying to Github Pages, and Derek Smart’s example configuration

For reference, here is the .travis.yml file for my own Jekyll site at the time of this writing.

With Travis and Github all set up, you now have a continuous integration and deployment pipeline that can publish a Jekyll site, free of Github’s restrictions, to Github Pages!