Scaling Beyond Lambda: Our Move to ECS and the Benefits for Enterprise Clients

The Problem: When Serverless Starts to Show Its Limits

Our journey began with AWS Lambda, a powerful tool for rapid deployment and effortless scaling. It was the perfect fit during our early stages, offering simplicity and low operational overhead. But as our system matured, cracks began to show.

• Cold Starts: Lambda’s cold start latency became noticeable in services that required low response times, especially under sporadic traffic.
  
  
• Resource Constraints: Memory and execution time limits restricted our ability to run more complex or longer-running workloads.
  
  
• Dependency Management: As our applications grew in complexity, managing large or specialized dependencies within Lambda functions became cumbersome.

As more customers came onboard to the FloQast platform—bringing increasingly large and complex datasets—we began to feel the strain of Lambda’s limitations more than ever.

An Example of Our Customer’s Pain

We ran into a critical issue when exporting datasets of around 5,000 items—our Lambda functions were consistently hitting memory limits, leading to sporadic 502 errors for customers. On top of that if they did succeed it would take an average of 18 seconds just to get the customer the report. With customer data volumes growing rapidly, we knew this problem would only get worse. It became clear that we needed a more scalable and resilient solution—and fast.

The Solution: Migrating to AWS ECS

To overcome the limitations we faced with AWS Lambda, we migrated our workloads to AWS ECS (Elastic Container Service). ECS offered the control, flexibility, and scalability we needed as our systems evolved.

Here’s how ECS solved our core challenges:

• No More Cold Starts: With ECS, our services run as long-lived containers, eliminating cold start latency entirely. This resulted in faster response times and improved user experience, especially for latency-sensitive endpoints.
  
• Flexible Resource Allocation: ECS allowed us to fine-tune CPU and memory configurations for each container. We could now run more demanding processes without worrying about hitting hard limits.‍
• Better Dependency Management: By containerizing our applications, we gained full control over the runtime environment. Complex dependencies and native libraries could be packaged once and run consistently across environments.

The Results: Successful ECS Migration

After careful planning and execution, we successfully migrated our critical services to AWS ECS. This wasn’t just a shift in infrastructure—it was an opportunity to modernize our codebase and deployment pipeline. The transition involved three key steps:

• Modernizing the Codebase: As part of the migration, we transitioned from JavaScript to TypeScript. This improved type safety, developer productivity, and long-term maintainability.
  
  
• Containerizing Services: We packaged our application into Docker containers. Not only did this improve quality by ensuring consistent runtime behavior across environments, but it also simplified local development setup and faster developer onboarding.
  
  
• Deploying to ECS: We set up ECS clusters and services, integrated them into our CI/CD pipeline, and thoroughly tested the new deployment process to ensure reliability and scalability.

As with any infrastructure change, this migration came with trade-offs. Some of the key challenges we faced included:

• Scaling Speed: One limitation we encountered with ECS is the slower response time when scaling under heavy load. When pushing the service to its limits, we observed noticeable delays in spinning up new instances to replace unhealthy ones. This introduces potential downtime during peak demand, and it’s something we’ll need to account for—especially as we prepare to onboard significantly larger customers.
• Increased Configuration Overhead: Unlike Lambda, which abstracts away most of the infrastructure details, ECS requires developers to manage and fine-tune a variety of settings—such as CPU/memory allocation, task definitions, networking modes, and scaling policies. This added complexity can slow down development and requires more operational awareness.

It didn’t take long to see the gains in both performance and stability! 🥳🎉

‍

Here are some key benefits we observed after the migration:

• Improved Performance: Elimination of cold starts resulted in significant latency reductions. We saw efficiency gains of up to 76% in export times! 🚀
• Increased Stability: Consistent resource availability led to more stable services (no more 502’s!). With no crashes seen in our logs since going live.
• Enhanced Management: Containerization simplified deployments and updates.

In conclusion, migrating from AWS Lambda to AWS ECS provided a robust and scalable foundation for our infrastructure—resolving key limitations and significantly improving system performance and manageability. More importantly, it positions us to confidently support the growing scale and complexity of enterprise customer data, both now and in the future.