NextJS and Rust: Creating a Custom Webpack Loader for Native Node Modules

Introduction

I needed a custom solution for loading a native node module in my nextjs project. The module was published with neon-bindings, and none of the below solutions worked for me:

• node-loader – Doesn’t work in nextjs because of the __dirname lookup. Look at this line. In my api route, module is requested relative to that module.

• webpack-asset-relocator-loader – Requires a hacky workaround because of a different path lookup is in dev/prod mode. I submitted an issue here for more details.

It’s not an issue in any of the loaders, or nextjs framework, I just needed to create a custom loader that would bootstrap the module from the correct lookup path. The module is used in node environment, so api routes and app page prop functions would work.

The module is a small slugifier utility that was written in Rust and interoperable with Node, I wrote about it in this article.

Code is available on Github and you can use the nextjs-node-loader. If you find it useful, give it a star on Github, thanks!

Creating a Custom Webpack Loader

What’s a loader

Webpack loader is a module that is responsible for transforming a specific type of file into a format that can be used by the JavaScript application. In my case, when a .node resource is found, loader appends a string to the output. When that part is evaluated, it will bootstrap a matched .node resource.

Step by step

I personally took node-loader as a template, but here I’ll extract the core parts needed for a loader:

npm install webpack loader-utils --save-dev

Obviously, webpack is a must. And loader-utils provide utility functions for webpack loaders.

And the loader part looks like this:

const { interpolateName } = require("loader-utils");

const schema = require("./options.json");

module.exports = function(content) {
  const { rootContext, _compiler, getOptions, emitFile } = this;
  const options = getOptions(schema);
  const { flags } = options;
  const outputPath = _compiler.options.output.path;

  const name = interpolateName(this, "[name].[ext]", {
    context: rootContext,
    content,
  });

  emitFile(name, content);

  return `
try {
  process.dlopen(module, ${JSON.stringify(
    outputPath
  )} + require("path").sep + __webpack_public_path__ + ${JSON.stringify(name)}${
    typeof flags !== "undefined" ? `, ${JSON.stringify(options.flags)}` : ""
  });
} catch (error) {
  throw new Error('nextjs-node-loader:\\n' + error);
}
`;
}

module.exports.raw = true;

Loader code explained

• Schema and interpolateName

The loader uses two external dependencies, loader-utils and options.json. loader-utils provides a helper function interpolateName which is used to create a name for the output file. options.json contains a schema for validating the options passed to the loader.

• Emit file

The emitFile function is called with the output filename and the content of the file, which tells webpack to emit the file to the output directory.

• The core functionality

The function returns a string that contains a try-catch block. This block attempts to load the emitted file using the dlopen function from the Node.js process module. The dlopen function takes two arguments: the first is the module to load, and the second is the path to the file to load.

The path module is used to construct the full path to the file, which includes the output path, the webpack public path, and the output filename. If the flags property is defined in the options object, it is also passed to dlopen as a third argument.

If an error occurs during the loading process, the catch block throws an error with a custom message that includes the name of this loader (nextjs-node-loader) and the error message.

• What’s an export raw?

The module.exports.raw = true line tells webpack that the loader returns raw binary data instead of a UTF-8 encoded string.

Include loader in a Next.js conf

And our loader is ready for use in a nextjs project by including it in webpack configuration.

module.exports = {
  webpack: (config, { dev, isServer, webpack, nextRuntime }) => {
    config.module.rules.push({
      test: /\.node$/,
      use: [
        {
          loader: "nextjs-node-loader",
        },
      ],
    });
    return config;
  },
};

If you create a local webpack loader for your webpack config, you would include it like:

{
    test: <regex>,
    use: [{ loader: path.resolve(__dirname, '<your_loader>')}],
}

Conclusion

I tried the loader in app page fetch data functions, api routes, prod/dev, it should work.

If it doesn’t work for you, please submit an issue on Github.

As for the article, while the code examples are focusing on a specific use case, I think that the principles of creating a custom loader can be applied to a wide range of use cases. With a good understanding of the Webpack loader API and some familiarity with Node.js and JavaScript, you can create a custom loader that transforms any type of file into a format that can be used by your application.