A single instance of a Node.JS application runs on only one thread and, therefore, doesn’t take full advantage of multi-core systems. And because of it Node.JS is not suitable for CPU/processor intensive tasks
Let’s assume we have a Node.JS application deployed in production, and its covered with extensive tests. But with few requests per second, the Node.JS process begins to leverage 100% of CPU, or there are some random spikes on a CPU graph, as a result, response time grows and affects all end-users. For sure you can just increase the amount of the running instances, but it’s not a solution to the problem, the application will behave the same.
What can cause this
CPU Intensive tasks, Loops and iterations: Any
.map,.reduce,.forEachand other iteration methods call that are CPU intensive can cause a problem if you don’t limit the size of the iterable collection. The same problem exist withforandwhileloops. If you have to deal with big collections, use streams or split collections into chunks and process them asynchronously. It will distribute the load between differentEventLoopiterations, and the blocking effect will be reduced.Recursive functions: Same principle here, you need to be aware of recursion depth, especially when the function is synchronous. For example: crawling large set of of url’s and processing huge chunk of files that we have downloaded from internet before saving them to disk. It may work fine until 100s are to be processed not 1000s. As a result, each call blocked the whole Node.JS process with 100% CPU load.
Large Payloads: Node.JS is created for dealing with a massive amount of asynchronous operations like making requests to databases or external API calls. And it works perfectly until payloads from external sources are small. Don’t forget that Node.JS needs to read a payload and store it in memory first, then parse JSON into an object (more memory added), perform some operations with the object.
Huge payloads from Node.JS services also can be a problem, because Node.JS stringifies objects to JSON first and then sends them to the client. All of these operations can cause high CPU load, make sure that payload size is not huge, use pagination, and don’t prepopulate unnecessary data. For GraphQL services use complexity to limit the response payload.
Promise.all:Promise.allis OK in itself. But it might cause problems if code is not written well or well thought out, or number of promises being processed are huge and process large datasets.Memory leaks:. Node.JS has a built-in Garbage Collector, depending on different conditions Garbage Collector removes unused objects from memory. Garbage collection is a costly operation. And if there is a memory leak in your Node.JS service, Garbage Collector will try to free memory over and over again without any success, just wasting CPU cycles and putting unnecessary loads.
How to find root cause
Finding is more difficult if project is already grown large (in terms of codebase). Obvious decision is trying to reproduce the issue locally first. Try to run your service locally, use htop or top to monitor uses and make some requests to it.
One can also use load testing packages on Node.JS, or use load-testing frameworks like Artillery or JMeter. Just remember local configuration should be as close to production as possible to reproduce it locally.
What about production? How to take CPU profile on running instance?
In most cases, it’s very difficult to reproduce performance issues, because you need the same environment setup; sometimes, the same data in databases, caches, and etc. A performance issue can be specific only for specific group of users because they have specific data.
Debug mode is not recommended to be enabled in production environment. As in debug mode Node.JS processes consume more resources, and it’s not safe. But there is a alternative approach, take profiles on-demand with inspector module Node.JS Inspector. It’s a Node.JS built-in module, you don’t have to install any additional dependencies, but it is recommended; one to use inspector-api. It’s a wrapper around inspector with promises support. Let’s create an endpoint that record a CPU profile, Let’s dive in and build an example
import { Controller, Post } from '@nestjs/common'
import { promisify } from 'util'
import Inspector from 'inspector-api'
const profileRecordTime = 10000
@Controller('/profile')
export class ProfileController {
@Post('/cpu')
async cpu() {
// don't wait till recording is finished
setImmediate(async () => {
// cpu profile will be saved in temp dir
const inspector = new Inspector({ storage: { type: 'fs' } })
// enabling and starting profiling
await inspector.profiler.enable()
await inspector.profiler.start()
// wait for 10 seconds and stop
await promisify(setTimeout)(profileRecordTime)
await inspector.profiler.stop()
console.log('CPU profile has been written')
await inspector.profiler.disable()
})
return true
}
}
Now once inspector is in place; it can be accessed at profile/cpu.
Note: Please do remember to secure inspector endpoint if you trying it on production or staging environment. Trying in production environment is not advised
All code is wrapped with setImmediate, because we don’t need to wait until the recording has ended. Let’s test it with curl:
curl -X POST http://127.0.0.1/profile/cpu
Now start accessing endpoint one might think creates this issue or better run a load testing profiler against that endpoint. And after some tries one should see a profile in the /temp directory
Let’s add a similar endpoint for heap profiling:
@Post('/heap')
async heap() {
// don't wait till recording is finished
setImmediate(async () => {
// cpu profile will be saved in temp dir
const inspector = new Inspector({ storage: { type: 'fs' } })
// enabling and starting profiling
await inspector.heap.enable()
await inspector.heap.startSampling()
// wait for 10 seconds and stop
await promisify(setTimeout)(profileRecordTime)
await inspector.heap.stopSampling()
console.log('Heap profile has been written')
await inspector.heap.disable()
})
return true
}
Now you can take CPU and heap profiles whenever you want, just copy them locally after recording.
If you don’t want to add this functionality as HTTP endpoints, you can wrap them inside process signal handlers instead, like this:
import { promisify } from 'util'
import Inspector from 'inspector-api'
const profileRecordTime = 10000
process.on('SIGUSR1', async () => {
const inspector = new Inspector({ storage: { type: 'fs' } })
await inspector.profiler.enable()
await inspector.profiler.start()
await promisify(setTimeout)(profileRecordTime)
await inspector.profiler.stop()
console.log('CPU profile has been written')
await inspector.profiler.disable()
})
process.on('SIGUSR2', async () => {
const inspector = new Inspector({ storage: { type: 'fs' } })
await inspector.heap.enable()
await inspector.heap.startSampling()
await promisify(setTimeout)(profileRecordTime)
await inspector.heap.stopSampling()
console.log('CPU profile has been written')
await inspector.heap.disable()
})
And use it by sending signals with the kill command:
kill -USR1 ${pid} // for CPU
kill -USR2 ${pid} // for Heap
If you use Kubernetes it might be tricky to copy files from production/staging, for this scenario inspector-api provides uploading profile files to AWS S3. To enable it, pass required options to the Inspector constructor and set AWS credential variables to the environment. Look at the following example
const inspector = new Inspector({
storage: {
type: 's3',
bucket: 'profileBucket',
dir: 'inspector'
}
})
Conclusion
Today we’ve discussed what can cause performance issues in Node.JS applications, how to find issues locally and in a running production/staging environment. IMHO performance fixes and optimization is the most interesting part in developers work life. If you have proper set of tools, it’s not so hard to find and fix issue.
Credits
Image Credits: Pexels
About The Author
I am Pankaj Baagwan, a System Design Architect. A Computer Scientist by heart, process enthusiast, and open source author/contributor/writer. Advocates Karma. Love working with cutting edge, fascinating, open source technologies.
To consult Pankaj Bagwan on System Design, Cyber Security and Application Development, SEO and SMO, please reach out at me[at]bagwanpankaj[dot]com
For promotion/advertisement of your services and products on this blog, please reach out at me[at]bagwanpankaj[dot]com
Stay tuned <3. Signing off for RAAM