Serverless Cold Starts ❤️ Dynamic Imports

Ah, let me regale you with a tale of JavaScript, that legendary speed demon of programming languages. And don’t worry, this is going to be a short one. And if this seems particularly obvious to you, hats off to you.

Picture this: You’re in corporate paradise, where the tech stack choices are as flexible as a brick wall and the infrastructure is handed down like ancient scrolls that must never be questioned. How thrilling!

Enter the magical world of “serverless” - a term that’s sprinkled around like fairy dust on every project. And with it comes its loyal companion: the cold start. Oh, the cold start! That delightful moment when your application decides to take a leisurely stretch before actually doing anything useful.

When can we go back to actual servers?

Now, for those blessed souls writing serverless code in a monolithic fashion (because why make things simple when they can be complicated?), you’ve probably witnessed the joy of watching your cold start times grow longer with each line of code you add. For those blissfully unaware of what I’m talking about - cherish your innocence, dear friends. Stay pure.

Heads up: The following is especially relevant for Firebase Cloud Functions users - this isn’t just about containerized deployments.

When I first encountered this phenomenon, I did what any reasonable developer would do: blamed it on either bloated JS dependencies (because who doesn’t have 47 packages to center a div?) or the hosting service being terrible. Plot twist: it could be both! Isn’t serverless architecture just delightful?

My primary hypothesis at this point is that our hosting provider was probably taking its sweet time spinning up the JS runtime, but faced with the fact that it is terribly unlikely that I will get to convince anyone to switch cloud providers on a whim, I decided to do the next best thing: I plastered console.log statements everywhere in the code, then dove into our staging container logs for answers. To my surprise, the container started up almost immediately after receiving the request from the client, absolving our cloud provider of all sins - which, sadly, meant the problem was lurking somewhere in our own code.

console.time("Import start");

import { abc } from "./api/abcHandler";
import { def } from "./api/defHandler";
import { ghi } from "./api/ghiHandler";
import { jkl } from "./api/jklHandler";
import { mno } from "./api/mnoHandler";
import { pqr } from "./api/pqrHandler";
import { stu } from "./api/stuHandler";
import { vwx } from "./api/vwxHandler";
import { yza } from "./api/yzaHandler";
import { bcd } from "./api/bcdHandler";
import { efg } from "./api/efgHandler";
import { hij } from "./api/hijHandler";
import { klm } from "./api/klmHandler";
import { nop } from "./api/nopHandler";
import { qrs } from "./api/qrsHandler";
import { tuv } from "./api/tuvHandler";
import { wxy } from "./api/wxyHandler";
import { zab } from "./api/zabHandler";
import { cde } from "./api/cdeHandler";
import { fgh } from "./api/fghHandler";
import { ijk } from "./api/ijkHandler";
import { lmn } from "./api/lmnHandler";
import { opq } from "./api/opqHandler";
import { rst } from "./api/rstHandler";
import { uvw } from "./api/uvwHandler";
import { xyz } from "./api/xyzHandler";
import { abc } from "./api/abcHandler";
import { def } from "./api/defHandler";
import { ghi } from "./api/ghiHandler";
import { jkl } from "./api/jklHandler";
import { mno } from "./api/mnoHandler";
import { pqr } from "./api/pqrHandler";

console.timeEnd("Import end");

const handlers: BaseContainerFunction[] = [
    new abc(),
    new def(),
    new ghi(),
    new jkl(),
    new mno(),
    new pqr(),
    new stu(),
    new vwx(),
    new yza(),
    new bcd(),
    new efg(),
    new hij(),
    new klm(),
    new nop(),
    new qrs(),
    new tuv(),
    new wxy(),
    new zab(),
    new cde(),
    new fgh(),
    new ijk(),
    new lmn(),
    new opq(),
    new rst(),
    new uvw(),
    new xyz(),
    new abc(),
    new def(),
    new ghi(),
    new jkl(),
    new mno(),
    new pqr(),
];

const server = new ContainerServer(handlers);
server.serve();

Brace yourself: this innocent-looking chunk of code (redacted for obvious reasons) was taking a jaw-dropping 10 seconds to evaluate, hogging the lion’s share of our 11-second average cold start time. Now, you might be thinking, “How were you not kicking up a fuss with pitchforks over this performance nightmare?” Well, here’s the thing - our containers were running like faithful workhorses around the clock, so this 11-second snooze fest only became relevant when we needed to rapidly scale up our instances. Otherwise, it was our dirty little secret, quietly tucked away in the corners of our production environment.

Most slow startup times in Firebase Cloud Functions can also likely be attributed to this issue.

const { abc } = require('./api')
const { def } = require('./api')
const { ghi } = require('./api')
...

exports.abc = abc
exports.def = def
exports.ghi = ghi
...

You get the idea.

For some reason, in my years of writing JavaScript (okay, fine, maybe it’s just been 3 years - but who’s counting?), I somehow never fully grasped that importing modules isn’t just a magical copy-paste operation. No, no - it’s a whole song and dance of code evaluation and mysterious Node.js wizardry that’s way above my pay grade. But of course it is! Just look at this beauty:

// greet.js
console.log("I am living in your walls");

// index.js
import "greet.js";

Run node index.js, and congratulations - you’ve just invited a creepy console message to take up permanent residence in your terminal.

This is why some people say web devs lack mechanical sympathy.

And it doesn’t just stop there. If your modules contains more imports, it will start recursively importing other modules (e.g., models, ORMs, secret managers, etc.) until it establishes a dependency graph.

Thankfully, the imports lead to modules defining services with a common interface:

// baseContainerFunction.ts
export abstract class BaseContainerFunction {
    protected abstract execute?(
        data: unknown,
        auth: Auth,
    ): Promise<object | void>;
}

// abcHandler.ts
import { AbcService } from "'~/services/abcService";

export class Abc extends BaseCloudFunction {
    protected override async execute(
        payload: AbcPayload,
        auth: Auth,
    ) {
        const service = new AbcService();
        return service.execute({
            ...payload,
            auth,
        });
    }
}

To break the chain of recursive imports, I just have to sprinkle some dynamic imports all over the codebase.

export class Abc extends BaseCloudFunction {
    protected override async execute(
        payload: AbcPayload,
        auth: Auth,
    ) {
        const { AbcService } = await import(
            "~/services/abcService"
        );
        const service = new AbcService();
        return service.execute({
            ...payload,
            auth,
        });
    }
}

And there you go. In theory, modules of services are lazily imported and would not be evaluated until a user calls the endpoint. Dynamically imported modules are cached after their first evaluation, so subsequent imports of the same module will reuse the cached instance rather than re-evaluating the module. Sure, a user might get a small delay if this is the first time an endpoint has been called since the container started, but I’m sure everyone can agree that this is a better tradeoff than the status quo. The optimized version executes in just 500ms, slashing the runtime by 95% from the original 11 seconds. And this is another reason why having a service layer is useful.

And let’s be honest here, while it’s fashionable to point fingers at Electron or whatever JavaScript framework is the villain of the week, the cold hard truth is that most performance nightmares can be traced back to us, the developers.

Though in our defense, JavaScript does seem to have a peculiar talent for making it surprisingly easy to shoot yourself in the foot.

And even if you aren’t particularly interested in serverless architecture, this same trick can be used to improve the start up times of CLIs written in JavaScript.

You could say that I am not really that much of a fan of Firebase in general. Rant incoming for my next blog post.