Netlify CMS + ManyToMany Relationship = .md file hell

When you configure your Netlify CMS schema for your Blog to work with many-to-many relationship between your Post entity and your Category entity, Netlify CMS creates one .md file per category.

In my case I´m using the @ncwidgets widget for Netlify CMS. It helps you to automatically create a UUID per entry that is required to make many-to-many referencing possible as we rely on a immutable ID to reference entities.

When dealing with tags or categories you often have rather many of them in your project, and you have to create all of them manually with the Netlify CMS dashboard.

This is not only time consuming, but it also will trigger one Netlify build per saved entry (except if you disable auto-building entirely).

Solution

Having a single, simple JSON file to manage your tags and use it as a source to auto-generate the Markdown category or tag files instead.

tags.json (source)

{
  "tags": [
    "Spring",
    "REST",
    "Java",
    "Pulumi",
    "Hibernate",
    "SaaS",
    "JPA",
    "Docker",
    "Kubernetes",
    "Cluster",
    "JavaScript",
    "Java",
    "AWS",
    "AWS Cognito",
    "Gridsome",
    "CI",
    "Devops",
    "Apollo",
    "Repository",
    "Resource"
  ]
}

Our goal is to generate a simple Markdown file for each tag having a unique id and a title for the category looking like this:

tag-gridsome.md (sample category / tag)

---

id: ctIVEVar0

title: Gridsome

---

For that task I´ve created the script below.

It esentially crawls the /content/tags directory for already existing tags as we always want to keep the existing ID´s in place that are referenced in the Blog Posts and creates a list of existing categories first.

Afterwards we go through each element we´ve defined in the tags.json file, check if it already exists and otherwise create it with a new UUID.

We always make sure that UUIDs are never clashing by holding a set of already used ones we always check each new ID against!

const fs = require("fs").promises;
const uuid = require("shortid");
const jdown = require('jdown');

const TAG_DIRECTORY = "tags";

const customTags = require("./tags.json").tags;

const convertTagToMarkdown = function (tag) {
  return "---\nid: " + tag.id + "\ntitle: " + tag.title + "\n---";
};

(async () => {
  const existingMarkdownTags = await jdown(TAG_DIRECTORY);

  /**
   * Prevention of duplicates
   */
  const tagIndex = new Set();
  const uuidIndex = new Set();

  /**
   * All-tags collection
   */
  const tags = [];

  /**
   * Make sure uuid is not clashing with existing ones
   */
  const getUUID = function () {
    let newID;
    do {
      newID = uuid();
    } while (uuidIndex.has(newID))
    return newID;
  }

  /**
   *
   */
  const createTagIfNotExists = function (tag) {
    if (!tagIndex.has(tag.toLowerCase())) {
      const id = getUUID();
      tagIndex.add(tag.toLowerCase());
      uuidIndex.add(id);
      tags.push({id, title: tag})
    }
  }

  /**
   * Read existing markdowns and register UUIDs
   */
  for (let key in existingMarkdownTags) {
    let tag = existingMarkdownTags[key];
    if (tag.id !== undefined) {
      uuidIndex.add(tag.id);
      tagIndex.add(tag.title.toLowerCase());
      tags.push(tag);
    } else createTagIfNotExists(tag.title);
  }

  /**
   * Create custom tags that
   * do not exist already
   */
  for (let key in customTags) {
    const tag = customTags[key];
    createTagIfNotExists(tag);
  }

  /**
   * Generate files from
   */
  for (let i = 0; i < tags.length; i++) {
    const tag = tags[i];
    let file = convertTagToMarkdown(tag);
    await fs.writeFile(TAG_DIRECTORY + "/tag-" + tag.title.toLowerCase() + ".md", file);
  }

})();

We then convert the list of existing and new tags back to markdown and write them back into the directory.

This solution has one drawback: you need to edit the .json file in your editor, but I still think it´s worth it if you have to manage hundrets of categories and you still are able to use the Netlify CMS Dashboard to add categories or tags manually if something is missing.

For testing purposes I´ve fetched all available Stackoverflow tags from here and converted them with the script into Markdown files as my Blog is tech oriented and I´m frequently searching for new related keywords.

My thought was it would be gread to have a pool of all relevant tech-oriented categories in my project to be used by the picker widget autocompletion in Netlify CMS, however loading 40.000+ categories as individual files in your project is a bad idea!

If still someone wants to enrich his site with tags from Stackoverflow as a source I can recommend pre-filtering it and better stay below 100 categories. Also keep in mind that each Markdown file will be processed by your static site generator.

The more you have the longer your built time will be

Sample project at Github