How to add sitemap, robots.txt and Google Analytics with Gatsby

What to do when Gatsby robot.txt and sitemap are not working correctly

Note: With Gatsby, apart from the usual npm install to add a plugin, we also need to add it to gatsby-config.js file.

SEO

Packages used: gatsby-plugin-sitemap andgatsby-plugin-robots-txt

We’ll need a sitemap and robots.txt. To verify if this already exists, go to {yoursite}.com/robots.txt. If not, here is how to set it up.

1. Install two packages with npm install gatsby-plugin-sitemap and npm install --save gatsby-plugin-robots-txt.
2. Edit gatsby-config.js to load these plugins as shown here
3. Set sitemap to the right address. This path was slightly different from the documentation.

plugins: [
  {
    resolve: 'gatsby-plugin-robots-txt',
    options: {
      host: 'https://www.example.com',
      sitemap: 'https://www.example.com/sitemap/sitemap-index.xml',
      policy: [{userAgent: '*', allow: '/'}],
    }
  }
]

4. This plugin only works in production mode. To test, run gatsby build && gatsby serve and access {yoursite}.com/robots.txt.

Google Analytics

Packages used: gatsby-plugin-google-gtag

1. Sign into Google Analytics, setup an account, get a tracking ID.
2. Add this to the gatsby-config.js file as shown here.

3. This plugin only works in production mode. To test your Global Site Tag is installed and firing events correctly, run gatsby build && gatsby serve.

In the next section, we take a look at Google Search Console, methods to verify ownership and why I had to addhead: true in the plugin config here.

Google Search Console

Google Search Console is a great tool to measure your site’s search traffic and performance, fix issues, and improve discoverability.

To get started, you need to verify ownership of the site. There are several methods to verify ownership — DNS verification for custom domain, uploading a file, Google Analytics, custom HTML tag etc.

For a Github site like mine, I chose URL prefix -> Other verification methods -> Google Analytics. For this to work, the analytics JS script must be placed in the header. To make that change, I had to update thegatsby-plugin-google-gtag config (from previous section) to set head: true.

And that should it! If you’re interested in the whole process of building a personal website with React, Github Pages and Gatsby — starting from goals, why I chose these tools, and learnings — I wrote about it detail here! 🙂