Tracking memory leaks in Node-based applications is not something you do every day. But when production starts eating its memory and alerts start firing, you need to be ready to roll up your sleeves and dig in.
Unfortunately, it usually is production that tells you. You would be very lucky to find a leak during development, where you have neither a monitoring tool nor enough load to notice.
How we found it
After a long weekend, our SRE team raised an alarm: the front-end application had almost run out of memory. The quick fix was to reboot the machines, but we knew we had a serious problem.
The leak had been there for a long time. We released at least once a day, and every release rebooted the machines, so the problem never had time to show. That week we released on Thursday and came back on Monday. Four days without a release was enough for memory to climb to alert level.
We quickly narrowed down roughly where the problem was. Because the leak predated the latest release, there was no point rolling back, so we kept operating as before and assigned an engineer to find it.
Setting up the investigation
Most articles on debugging Node memory leaks use Chrome DevTools to record and analyze memory. In our case recordings took too long, some never finished, and some crashed Chrome. So we tried something else.
We used node-heapdump, a package that writes a heap dump of a V8-based application for later inspection. We were on an old version of Next.js at the time, so we added a route we could call manually to write a heap dump to disk each time it was hit.
To inspect a dump, open chrome://inspect, choose "Open dedicated DevTools for Node", go to the Memory tab and load the .heapsnapshot file. You can then inspect every object in the heap, see what references it, and find the file it came from.
Recreating the problem
Next.js pre-renders every page on the server, so every visit allocated resources on our servers. With a leak in place, we were in a vicious circle: we wanted more users, and more users meant the servers ran out of memory faster.
Since production had exposed the problem, we made development as close to production as possible. We ran a production build and then added load: a small Python script that opened a configurable number of Chrome instances and visited our homepage, simulating real traffic.
We took three heap snapshots:
- right after the application started
- after the first round of simulated load
- after the second round
Memory grew with every snapshot. There was the leak.
Finding the cause
To see what had grown, select the largest snapshot, switch to Comparison mode, and compare it against the smallest.
The biggest difference was in closures: events we had subscribed to and never unsubscribed from. Digging through the largest groups of objects kept leading us to Axios objects, and inspecting those one by one showed that many were created by the Contentful SDK.
We used Contentful heavily, so it was a strong suspect. A quick look at the code confirmed it: we were creating a new Contentful client for every request.
The fix
Our first idea was a singleton: create the client once and reuse it. It did not work, because we used several Contentful spaces, and a client created for one space cannot read from another.
The fix was an object pool of singletons: one client instance per Contentful space, reused across requests.
We ran the experiment again. This time the three snapshots were almost identical, which meant the application was no longer allocating memory with every visit.
Because the leak was found in production, we had to confirm the fix there too. We deployed the next day and watched memory closely. After 24 hours there was no increase, but we let it run for several days before declaring victory. A week later, New Relic showed a flat memory line where there had been a steady climb.
What we took away
Engineers often do not think about monitoring, and forget to give monitoring tools the information they need to profile the code.
Our work is not done when code reaches production. We are responsible for watching it there, measuring its impact and improving it. Code that serves a few users and code that serves millions are not the same thing, and the lessons from monitoring are some of the most valuable we get.
In this case monitoring was critical. Without it, the application would have crashed for a few hundred thousand users.
Memory leaks are tricky, but there is real satisfaction in tracking one down. It pushes you to understand how your tools work inside, not just how to use them. Do not run away from tasks like this.