How to parse MDX in Next.js using TypeScript?
Yet another strategy to store and parse MDX metadata in Next.js using TypeScript and without using any external database/CMS.
How to parse MDX in Next.js using TypeScript?While creating this blog, I was stuck in a situation where I was unable to extract the metadata from the markdown (MDX) files. I needed the metadata to create a database/registry of posts which I could use to search, list and filter the posts. I did not want to use next-mdx-remote, as it was still an experimental feature while writing this blog and I didn't want to overcomplicated the project.
So, without complicating it too much I decided to rearrange the directory structure of the posts like so:
Directory Structure
I went for simple directory structure, where the metadata is stored in a separate file and the markdown file is stored in the same directory. "The directory name becomes the slug of the post". I don't have the worry about the uniqueness of the slugs because the filesystem won't allow me to create a file with the same name. Also, /app routing from NextJS works out of the box with this structure.
Directory-Structure
📂devy.in
├──📂 src
| ├──📂 app
| | ├──📂 post
| | | |──📂 my-first-blog
| | | | ├──📘 metadata.ts
| | | | ├──📜 page.mdx
| | | |──📂 another-blog
| | | | ├──📘 metadata.ts
| | | | ├──📜 page.mdx
| | | |──📂 yet-another-blog
| | | | ├──📘 metadata.ts
| | | | ├──📜 page.mdx
| | | |──📘 layout.tsx
| | | |──📘 page.tsx
| | |──📂 types
| | | ├──📘 metadata.types.ts
| |──📂 library
| | ├──📂 utils
| | | |──📘 crawler.ts
| | | |──📘 postsRegistry.ts
| | ├──📂 logger
| | | |──📘 logger.ts
| |── 📘 mdx-components.tsx
|──🗒️.env
|──🗒️.gitignore
|──📒 next.config.js
|──📒 package.json
|──📜 README.md
|──📒 tsconfig.json
There is room for improvement for sure, but let's keep it simple for this tutorial
Contents of relevant files
metadata.types This file contains the type of the metadata object in addition to categories list. Static list of categories prevents from spamming the category/tags list. I can always come back and add more categories later.
Also, i have extended MetaData type from next.js to add the my custom properties such as slug, categories, published etc. Moreover, I can simply loop through this list to generate SSR category pages using generateStaticParams.
/src/app/types/metadata.types.ts
import { Metadata } from "next";
export const CATEGORIES = [
"blog",
"react",
"typescript",
"nextjs",
"personal",
"life",
];
export type Category = (typeof CATEGORIES)[number];
export interface MetaData extends Metadata {
author: string;
createDate: Date;
updateDate?: Date;
categories: Category[];
slug: string;
published?: boolean;
}
~my-first-blog/metadata.ts This file contains the metadata for each post.
/src/app/post/my-first-blog/metadata.ts
import { MetaData } from "@/app/types/metadata.types";
import dayjs from "dayjs";
const slug = "my-first-blog";
export const metadata: MetaData = {
author: "My Name",
title: "My First Blog",
slug,
description:"My First Blog using Next JS",
createDate: dayjs("01-May-2024", "DD-MMM-YYYY").toDate(),
updateDate: dayjs("03-May-2024", "DD-MMM-YYYY").toDate(),
categories: ["blog", "nextjs"],
published: true,
};
~my-first-blog/page.mdx This file contains our blog in markdownX language.
/src/app/post/my-first-blog/page.mdx
import { metadata } from './metadata';
export { metadata };
## {metadata.title}
##### {metadata.description}
***
My First Blog using Next JS 🥳🎈🎉
👉🏻 Demo
How do I use the metadata.ts file?
Now that we have the basic structure of the blog, we need to somehow do the following:
/src/library/utils/crawler.ts
import { MetaData } from "@/app/types/metadata.types";
import { readFile, readdir } from "fs/promises";
import path from "path";
import ts from "typescript";
import logger from "../logger/logger"; // Ignore this. Just use console.log() instead
const POST_DIRECTORY = path.join("src", "app", "post");
const METADATA_FILE = "metadata.ts";
const crawlerLogger = logger("crawler.ts");
export const readAllPosts = async () => {
// 1. Read all the folders within the /src/app/post directory.
const postDirectories = await readdir(POST_DIRECTORY, {
withFileTypes: true,
});
const dirNodes = postDirectories
.filter((dirent) => !dirent.name.startsWith("_")) // Ignores any _drafts folder
.filter((dirent) => !dirent.isFile()); // Ignores any page.tsx or layout.tsx files in the directory
// 2. Read the `metadata.ts` file from each folder.
const readAllMetaDataAsync = dirNodes.map(async (dirent) => {
const post = await readFile(
path.join(dirent.path, dirent.name, METADATA_FILE),
"utf-8"
);
// 3. Somehow transpile the typescript code within `metadata.ts` to javascript.
const jsCode = ts.transpile(post, { esModuleInterop: true })
// 4. Evaluate the transpiled javascript to a `metadata` object.
return eval(jsCode);
});
const allPosts = await Promise.all(readAllMetaDataAsync);
crawlerLogger.log(`All Posts read. [allPosts.length=${allPosts.length}]`);
// 5. Return the objects as a array: `Metadata[]`.
return allPosts as MetaData[];
};
In the above code I used typescript to transpile the typescript code to javascript. ts.transpile() takes in the typescript code as a string and returns the transpiled javascript code as a string. The key gotcha in this section is to pass { esModuleInterop: true } as one of the TranspileOption so that the TS compiler can understand the module imports in the TS file headers. Next, I used eval to evaluate the transpiled javascript code to a metadata object.
In the next section, we will use the metadata exported to create a simple blog list and also see how can we filter the posts based on the category.
