Express.js to Fastify Migration: Performance Gains and Gotchas

Migrating a core service is always a daunting task, but the potential performance gains from switching frameworks can be well worth the effort. At MisuJob, we recently undertook a significant migration, moving one of our key backend services from Express.js to Fastify. This post details our experience, focusing on the performance benefits we observed, the challenges we encountered, and the strategies we employed to ensure a smooth transition.

Why Fastify? Performance, Performance, Performance

Our primary motivation for considering Fastify was performance. As MisuJob processes 1M+ job listings daily and performs complex, AI-powered job matching, even minor performance improvements can translate to significant cost savings and a better user experience. Express.js, while mature and widely adopted, has known performance limitations, especially when handling high concurrency. Fastify, on the other hand, is built with a focus on speed and low overhead. Its architecture leverages a Just-In-Time (JIT) compiler and optimized routing, resulting in substantially faster request processing.

We hypothesized that Fastify’s superior performance would allow us to:

• Reduce server resource consumption.
• Lower response times for critical API endpoints.
• Improve the overall scalability of our platform.

Before committing to a full migration, we conducted extensive benchmarking. We created a representative workload mimicking the traffic patterns of our production environment and ran it against both Express.js and Fastify instances. The results were compelling.

Here’s a simplified example of the benchmark setup we used with autocannon:

autocannon -c 100 -d 10 -p 10 localhost:3000/api/jobs

The results consistently showed Fastify handling significantly more requests per second (RPS) with lower latency. We saw an average increase of 30-40% in RPS with Fastify compared to Express.js under the same load conditions. This initial validation gave us the confidence to proceed with the migration.

The Migration Process: A Step-by-Step Guide

The migration process was carefully planned and executed in stages to minimize disruption.

1. Project Setup and Core Dependencies

We started by creating a new Fastify project and installing the necessary dependencies. Fastify’s plugin system makes it easy to integrate existing middleware and functionality. We aimed to replicate the functionality of our Express.js service as closely as possible in the new Fastify environment.

// Example: Registering a Fastify plugin
const fastify = require('fastify')({ logger: true })

fastify.register(require('fastify-cors'), {
  origin: '*' // Adjust this in production!
})

fastify.get('/api/jobs', async (request, reply) => {
  // ... your job listing logic here
  return { message: 'Hello Fastify!' }
})

fastify.listen({ port: 3000 }, (err, address) => {
  if (err) {
    fastify.log.error(err)
    process.exit(1)
  }
  fastify.log.info(`Server listening on ${address}`)
})

2. Route Migration and Validation

The most time-consuming part of the migration was porting our existing routes from Express.js to Fastify. While the basic routing concepts are similar, Fastify’s request handling differs slightly. One key difference is Fastify’s emphasis on asynchronous code and promises. We rigorously tested each route after migration to ensure it behaved identically to its Express.js counterpart.

3. Database Integration and Optimization

Our service relies heavily on database interactions. We used PostgreSQL for our primary data store. We carefully reviewed our database queries to ensure they were optimized for performance in the new Fastify environment. We also leveraged Fastify’s built-in support for connection pooling to improve database connection management.

To optimize queries, we used EXPLAIN ANALYZE to understand the query plans. For example:

EXPLAIN ANALYZE SELECT * FROM job_listings WHERE company_id = 123 AND location LIKE '%Berlin%';

Analyzing the output helped us identify missing indexes and inefficient query patterns. We added an index on company_id and rewrote the location filter to use a more precise match instead of LIKE '%Berlin%'. This drastically improved query performance.

4. Authentication and Authorization

We reimplemented our authentication and authorization mechanisms using Fastify’s plugin system. We used JSON Web Tokens (JWTs) for authentication and role-based access control for authorization. We ensured that the authentication process was as efficient as possible to minimize its impact on overall performance.

5. Monitoring and Logging

We integrated our existing monitoring and logging infrastructure with the new Fastify service. We used Prometheus and Grafana to track key performance metrics, such as request latency, error rates, and resource utilization. We also configured detailed logging to help us identify and troubleshoot any issues that arose.

6. Canary Deployment and Rollout

Before fully migrating to Fastify, we performed a canary deployment. We routed a small percentage of traffic to the Fastify service while the majority of traffic continued to be served by the Express.js service. This allowed us to monitor the performance and stability of the Fastify service in a real-world production environment. After a successful canary deployment, we gradually increased the percentage of traffic routed to Fastify until it was handling all requests.

Performance Gains: Real Numbers from Production

After the migration, we observed significant performance improvements in our production environment. The following table summarizes the key performance gains:

These numbers represent real-world performance improvements that have translated to tangible benefits for our platform and our users. The reduction in request latency has improved the responsiveness of our API endpoints, leading to a better user experience. The lower CPU utilization has allowed us to consolidate our server infrastructure, resulting in cost savings. The reduced error rate has improved the overall stability of our platform.

Challenges and Gotchas: Lessons Learned

While the migration was successful overall, we encountered several challenges along the way:

• Plugin Compatibility: Not all Express.js middleware has a direct equivalent in Fastify. We had to find alternative plugins or write our own custom plugins to replicate certain functionality.

• Asynchronous Code: Fastify heavily relies on asynchronous code. We had to refactor some of our existing code to use async/await patterns to ensure compatibility. This required careful attention to error handling and promise management.

• Ecosystem Maturity: While Fastify’s ecosystem is growing rapidly, it is not as mature as Express.js’s. We sometimes had to spend more time researching and troubleshooting issues due to the smaller community size.

• Serialization: Fastify’s default serialization strategy can be strict. We had to configure custom serializers for certain data types to ensure compatibility with our existing API contracts.

Salary Insights: Impact of Backend Performance on Compensation

We’ve observed a strong correlation between backend performance and the salaries offered to backend engineers, particularly in performance-critical roles. Companies that prioritize performance optimization often pay more to attract and retain engineers with expertise in this area.

Here’s a table illustrating typical salary ranges for Senior Backend Engineers in various European countries, taking into account their experience with high-performance frameworks like Fastify:

These figures are based on MisuJob’s internal data and reflect the current market conditions. As you can see, proficiency in technologies that contribute to improved backend performance can significantly impact your earning potential.

Key Takeaways

Our migration from Express.js to Fastify has been a resounding success. We have observed significant performance improvements that have translated to tangible benefits for our platform and our users. While the migration process was not without its challenges, we learned valuable lessons that will inform our future development efforts.

Here’s a summary of our key takeaways:

• Performance is paramount: Fastify’s focus on performance makes it an excellent choice for high-traffic, low-latency applications.
• Plan carefully: A well-defined migration plan is crucial for minimizing disruption and ensuring a smooth transition.
• Test thoroughly: Rigorous testing is essential for validating the correctness and performance of the new service.
• Monitor closely: Continuous monitoring is necessary for identifying and resolving any issues that arise in production.
• Invest in your team: Training and development are essential for equipping your team with the skills needed to work with new technologies.

We believe that Fastify is a compelling alternative to Express.js for Node.js backend development. Its superior performance, combined with its modern architecture and growing ecosystem, makes it a strong contender for building scalable and efficient web applications. We encourage other engineering teams to consider Fastify as a viable option for their projects.