Node.js PM2 Infinite Restart Loop & Memory Leak Troubleshooting (macOS)

Diagnose and resolve Node.js PM2 processes repeatedly crashing due to memory leaks on macOS. Understand root causes and implement robust fixes.

A Node.js application running under PM2 in a macOS local development environment can sometimes enter an infinite restart loop, often signaling an underlying memory leak or unhandled exception. This guide provides a highly technical approach to diagnose and resolve such critical issues, drawing on 16 years of web hosting and DevOps expertise.

Symptom & Error Signature

Users typically observe their Node.js application becoming unresponsive or unavailable, with PM2 reporting frequent restarts. The application process's memory usage might steadily climb until it crashes, or it might crash immediately due to an uncaught error.

Common pm2 status Output:

❯ pm2 status
┌────┬────────────────────┬───────────┬──────┬─────────┬─────────┬───────────┬────────────────┬───────────────────┐
│ id │ name               │ namespace │ mode │ pid     │ uptime  │↺    │ status    │ cpu      │ mem               │
├────┼────────────────────┼───────────┼──────┼─────────┼─────────┼───────────┼────────────────┼───────────────────┤
│ 0  │ my-node-app        │ default   │ fork │ 45678   │ 0s      │ 153 │ errored   │ 0%       │ 0B                │
└────┴────────────────────┴───────────┴──────┴─────────┴─────────┴───────────┴────────────────┼───────────────────┘
# The '↺' column (restarts) will show a rapidly increasing number.
# 'status' will frequently cycle through 'stopping', 'stopped', 'errored', 'launching'.

Typical pm2 logs Output Indicating Memory Issues:

❯ pm2 logs my-node-app --lines 50
...
0|my-node-app | <--- Last few GCs --->
0|my-node-app |
0|my-node-app | [45678:0x103000000] 17940 ms: Scavenge 1283.4 (1337.3) -> 1279.9 (1338.3) MB, 3.4 / 0.0 ms (average mu = 0.160, current mu = 0.066) allocation failure
0|my-node-app | [45678:0x103000000] 17944 ms: Scavenge 1284.1 (1338.3) -> 1280.7 (1339.3) MB, 2.7 / 0.0 ms (average mu = 0.160, current mu = 0.066) allocation failure
0|my-node-app |
0|my-node-app | <--- JS stacktrace --->
0|my-node-app |
0|my-node-app | FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
0|my-node-app |     1: 0x10003b575 node::Set0FlagsAndLog (node::StartInspector(v8::Isolate*, int, std::unique_ptr<node::inspector::InspectorSocketServer>, std::unique_ptr<node::inspector::InspectorSocketServer>)) [node]
...
# OR, indicating an unhandled error leading to crash:
0|my-node-app | (node:45678) UnhandledPromiseRejectionWarning: Error: connect ECONNREFUSED 127.0.0.1:5432
0|my-node-app |     at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16)
0|my-node-app | (Use `node --trace-warnings ...` to show where the warning was created)
0|my-node-app | (node:45678) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch().

Root Cause Analysis

The core of an infinite restart loop with PM2 often lies in the application's runtime behavior, exacerbated by PM2's resilience features.

1. Memory Leaks: This is the most common cause. Node.js applications, while garbage-collected, can suffer from memory leaks if objects are inadvertently retained in memory, preventing the garbage collector from reclaiming their space.
   • Persistent references: Global caches without eviction policies, objects held by closures, or improperly managed event listeners.
   • Unclosed resources: Database connections, file handles, network sockets, or streams that are opened but never closed.
   • Third-party libraries: Bugs or improper usage of external modules can introduce leaks.
2. Unhandled Exceptions & Promise Rejections: If the application frequently encounters uncaught errors or unhandled promise rejections, the Node.js process will terminate. PM2, detecting a crash, will immediately restart the process, leading to a loop if the error is persistent.
3. Resource Exhaustion (non-memory):
   • File Descriptor Limit (EMFILE): Opening too many files or network sockets without closing them can exceed the operating system's file descriptor limit, causing new operations to fail and potentially crash the application.
   • Excessive CPU Usage: An infinite loop or highly inefficient code can consume 100% CPU, making the application unresponsive and potentially triggering OOM (Out Of Memory) killer if memory climbs due to side effects.
4. Incorrect PM2 Configuration:
   • max_memory_restart: A poorly configured max_memory_restart limit might be too low, causing PM2 to restart the app proactively even for normal, high memory usage, rather than addressing the root leak.
   • instances: Running too many instances for the available system RAM can lead to all processes collectively hitting memory limits.

Step-by-Step Resolution

Debugging memory leaks and infinite loops requires a methodical approach, often involving code inspection and advanced profiling.

1. Initial Diagnosis: Check PM2 Status and Logs

Start by thoroughly examining the PM2 logs and real-time monitoring. This provides the first clue about the nature of the crash.

# Check PM2 status for restart count and current state
pm2 status

# Tail PM2 logs for your application, looking for error messages
# Pay close attention to messages preceding the 'errored' state or repeated warnings.
pm2 logs my-node-app --lines 100 --timestamp

# Use PM2's interactive monitor to observe CPU and Memory usage in real-time
pm2 monit

# On macOS, use Activity Monitor or 'top' to check overall system resource usage
# (GUI)
open /System/Applications/Utilities/Activity Monitor.app
# (CLI)
top -o rsize # Order by resident memory size

2. Analyze Node.js Heap & CPU Profile

If logs indicate "JavaScript heap out of memory" or memory constantly climbs, a memory leak is highly probable. Node.js's V8 engine has powerful debugging tools.

# Option 1: Using Node.js's built-in Inspector for real-time analysis

# 1. Stop your PM2 process to prevent automatic restarts during manual debugging.
pm2 stop my-node-app

# 2. Modify your PM2 ecosystem file (e.g., ecosystem.config.js) to enable the inspector.
#    This allows PM2 to manage the process while you debug.
#    Alternatively, for a one-off debug, you can run `node --inspect app.js` manually.
#    Make sure the port (9229 in this example) is not in use.

// ecosystem.config.js example with inspector enabled
module.exports = {
  apps: [{
    name: "my-node-app",
    script: "./app.js",
    node_args: "--inspect=0.0.0.0:9229", // Listen on all interfaces, specify port
    instances: 1, // Only one instance for debugging
    exec_mode: "fork",
    // Remove or temporarily disable max_memory_restart during debugging
    // max_memory_restart: "500M",
    env: {
      "NODE_ENV": "development"
    }
  }]
};

# 3. Start the application with PM2 using the updated ecosystem file.
pm2 start ecosystem.config.js

# 4. Find the debug URL from PM2 logs or `pm2 desc my-node-app`.
#    Example from logs: `Debugger listening on ws://127.0.0.1:9229/<uuid>`
# 5. Open Google Chrome. Navigate to `chrome://inspect`.
# 6. Under "Remote Target", you should see your Node.js process. Click "inspect".
# 7. In the DevTools window:
#    - Go to the **Memory** tab.
#    - Take heap snapshots (`Take snapshot`) at different points in your application's lifecycle, especially after actions that might trigger a leak. Compare snapshots to identify objects that are growing in count or size.
#    - Go to the **Performance** tab.
#    - Record a CPU profile (`Record`) during periods of high CPU usage to identify performance bottlenecks.

# Option 2: Using the `heapdump` library for programmatic snapshots (if needed for specific events)
# 1. Install the `heapdump` package:
npm install heapdump

// In your application code (e.g., app.js or a debug utility file)
// Add this only for development/debugging, remove for production.
if (process.env.NODE_ENV === 'development') {
    const heapdump = require('heapdump');
    // You can trigger a heap dump on a signal (like SIGUSR2) or programmatically
    process.on('SIGUSR2', () => {
        console.log('Received SIGUSR2, taking heap dump...');
        heapdump.writeSnapshot((err, filename) => {
            if (err) console.error('Error writing heapdump:', err);
            else console.log('Heap dump written to', filename);
        });
    });
}

# 2. Start your app with PM2:
pm2 start app.js --name my-node-app

# 3. Trigger a heap dump by sending the SIGUSR2 signal to the Node.js process:
kill -SIGUSR2 $(pm2 pid my-node-app)

# 4. Analyze the generated `.heapsnapshot` file using Chrome DevTools (Memory tab -> Load profile).

Heap snapshots and CPU profiles can be very large. Ensure your macOS environment has sufficient disk space before generating multiple snapshots. Running profiling tools can also significantly impact application performance.

3. Review Code for Common Memory Leak Patterns

Based on the profiling results, focus your code review on areas identified as problematic.

• Event Emitters: Ensure listeners are removed when no longer needed using removeListener or removeAllListeners. Using once is a good practice for single-shot events.
• Closures: Be cautious of closures that unintentionally capture large objects from their outer scope, preventing garbage collection.
• Global Caches/Arrays: Implement strict eviction policies (e.g., LRU cache, time-based expiry) for global data structures that store growing amounts of data.
• Streams: Always ensure Node.js streams are consumed, piped to a destination, or explicitly destroyed when finished. Unconsumed readable streams can buffer data indefinitely.
• Database/Network Connections: Properly close connections after use, or ensure connection pools are configured with appropriate limits and idle timeouts.
• setInterval/setTimeout: Always clear timers with clearInterval or clearTimeout when the repeating or delayed task is no longer required.

// Example: Common Event Listener Leak Scenario
const EventEmitter = require('events');
const eventBus = new EventEmitter();
let userSessions = []; // Global store

function createUserSession(socket) {
  const session = { id: Math.random(), socket: socket };
  userSessions.push(session);

  // BAD: Listener added on every new session/connection, never removed.
  // Each `session` object (potentially large due to `socket`) is retained
  // by its closure inside this new listener, preventing GC.
  eventBus.on('dataReceived', (data) => {
    // Process data for this specific session
    console.log(`Session ${session.id} received data: ${data}`);
  });

  // This is how it should ideally be handled, or use `socket.on('data', ...)` directly
  // GOOD: A single handler for all events, or ensure listeners are removed.
  socket.on('close', () => {
    // Remove session from global store
    userSessions = userSessions.filter(s => s.id !== session.id);
    // Crucially, remove the specific listener if it was bound uniquely per session
    // This often requires named functions or binding context carefully.
  });
}

4. Configure PM2 for Memory Management

While fixing the leak is paramount, PM2 can mitigate system impact by restarting processes before they exhaust all memory.

// ecosystem.config.js example for PM2 with memory limit
{
  "apps" : [
    {
      "name"         : "my-node-app",
      "script"       : "./app.js",
      "instances"    : 1, // Start with 1 instance during dev to simplify debugging
      "exec_mode"    : "fork",
      "watch"        : false, // Disable watch during debugging to avoid extraneous restarts
      "max_memory_restart": "400M", // Restart if memory exceeds 400MB (adjust based on app's stable footprint)
      "env": {
        "NODE_ENV": "development"
      },
      "env_production" : {
        "NODE_ENV": "production"
      }
    }
  ]
}

# Apply the updated PM2 configuration
pm2 reload ecosystem.config.js

max_memory_restart is a critical mitigation strategy for production but does not solve the underlying memory leak. In development, it helps stabilize your system while you diagnose the root cause.

5. Handle Uncaught Exceptions & Promise Rejections Gracefully

Ensure your application has robust error handling to prevent unexpected process termination.

// In your main application file (e.g., app.js)

// Global Uncaught Exception Handler (synchronous errors)
process.on('uncaughtException', (err) => {
  console.error('UNCAUGHT EXCEPTION! Shutting down...', err.stack);
  // Log detailed error information to a file or a monitoring service
  // Perform any critical cleanup (e.g., closing database connections)
  // In production, exiting gracefully is often recommended to prevent corrupted state.
  process.exit(1); // Exit with a failure code
});

// Global Unhandled Promise Rejection Handler (asynchronous errors)
process.on('unhandledRejection', (reason, promise) => {
  console.error('UNHANDLED REJECTION! Shutting down...', reason, promise);
  // Log the rejection reason and the promise object
  // For development, just logging might be enough. In production, consider exiting.
  // If the unhandled rejection indicates a critical failure, exit to ensure clean restart.
  // process.exit(1);
});

// For specific async functions, always use try-catch or .catch()
async function fetchData() {
  try {
    const data = await someAsyncOperation();
    // Process data
  } catch (error) {
    console.error('Error in fetchData:', error);
    // Handle specific error, maybe retry or return default
  }
}

While catching uncaughtException can prevent an immediate crash, it can leave your application in an undefined or corrupted state. In production, it's generally recommended to log the error, perform minimal cleanup, and then exit the process (e.g., process.exit(1)) to ensure a clean restart by PM2 or your orchestrator.

6. Increase File Descriptor Limit (if EMFILE observed)

If pm2 logs or your application logs show Error: EMFILE, too many open files, you've hit the operating system's file descriptor limit.

# Check current open file limit for the shell
ulimit -n

# Temporarily increase the limit for the current shell session
# This applies to processes launched from this terminal session, including PM2.
ulimit -n 8192 # Set to a higher value, e.g., 8192 or 16384

# Then restart your PM2 application from this shell
pm2 restart my-node-app

For persistent changes on macOS:

1. Edit /etc/sysctl.conf:

   sudo nano /etc/sysctl.conf
   

   Add or modify these lines:

   kern.maxfiles=65536
   kern.maxfilesperproc=65536
   

   Apply the changes:

   sudo sysctl -p /etc/sysctl.conf
   

2. Edit launchd limits (for system-wide effect, more complex): Create/edit /Library/LaunchDaemons/limit.maxfiles.plist (for system-wide applications) or ~/Library/LaunchAgents/limit.maxfiles.plist (for user-specific applications). This is a more involved process and often unnecessary for local development if ulimit is handled in your shell config.

On production Linux environments (e.g., Ubuntu/Debian), these limits are typically managed via /etc/security/limits.conf for user sessions or directly within Systemd service unit files for services.

7. Update Node.js and PM2

Ensure you're running the latest stable versions of Node.js and PM2 to benefit from bug fixes, performance improvements, and security patches.

# Update Node.js (using nvm - recommended on macOS)
nvm install --lts # Installs the latest LTS version
nvm use --lts     # Switches to it
nvm alias default 'lts/*' # Set as default for new shells

# Update PM2 (globally installed)
npm install pm2@latest -g
pm2 update # This updates the PM2 daemon itself

8. Consider Containerization for Production Readiness

While debugging on macOS, remember that production environments typically use containerization (Docker) and orchestration (Kubernetes) for robust deployments.

# Example Dockerfile for a Node.js PM2 application
FROM node:lts-alpine # Use a lean base image

WORKDIR /app

# Copy package.json and install dependencies first for better caching
COPY package*.json ./
RUN npm install --production --unsafe-perm=true --allow-root # --unsafe-perm/--allow-root for Docker builds

COPY . . # Copy remaining application code

# PM2 recommends `pm2-runtime` for Docker environments
CMD ["pm2-runtime", "start", "ecosystem.config.js", "--env", "production"]

# Build and run the Docker image (for local testing or production deployment)
docker build -t my-node-app-image .
docker run -d --name my-node-app 
  --memory="1g"  # Set a memory limit (e.g., 1GB)
  --cpus="0.5"   # Limit CPU usage (e.g., 0.5 cores)
  -p 3000:3000   # Map port
  my-node-app-image

Docker and container orchestration tools (like Kubernetes) provide excellent resource isolation and management features, which are crucial for preventing a single misbehaving application from affecting the entire host system in a production environment. Always monitor container-level metrics for memory and CPU usage.