AWS Expands Lambda Capabilities with New Provisioned Mode for SQS Event Source Mapping
In a significant development for cloud computing, Amazon Web Services (AWS) has unveiled the general availability of a new feature designed to enhance the performance of event-driven applications. This new feature is the provisioned mode for AWS Lambda in conjunction with Amazon Simple Queue Service (Amazon SQS) Event Source Mapping (ESM). It aims to offer users better control over their applications by providing dedicated polling resources, ultimately leading to improved throughput. With this update, AWS users can now experience up to three times faster scaling and 16 times increased concurrency, allowing for the processing of events with reduced latency and more effective handling of sudden traffic spikes.
Event-driven architectures have become a cornerstone of modern applications, facilitating communication between services through events and messages. Amazon SQS is frequently utilized as an event source for Lambda functions, enabling developers to create scalable and loosely coupled applications. Despite the automatic queue polling and function invocation capabilities of SQS ESM, many AWS users have expressed the need for more granular control over polling behavior to better manage variable traffic patterns and ensure low processing latency.
New Provisioned Mode: Enhanced Performance and Control
The newly introduced provisioned mode for SQS ESM addresses these demands by implementing event pollers. These are dedicated resources optimized to manage anticipated traffic patterns more efficiently. The provisioned mode allows for the scaling of event pollers up to 1,000 concurrent executions per minute—more than three times faster than previous capabilities. This feature can handle up to 20,000 concurrent executions, offering 16 times the capacity to process millions of events using Lambda functions. Such improvements ensure that even during traffic surges, applications can maintain low latency predictably.
Industries ranging from financial services to gaming are actively leveraging AWS Lambda with Amazon SQS to process real-time events critical to their operations. For these organizations, including leading online gaming platforms and financial institutions, ensuring consistent sub-second processing times during peak periods is crucial. The provisioned mode for SQS ESM is a strategic tool to meet these stringent performance criteria while maintaining cost efficiency.
Detailed Control and Performance Optimization
Provisioned mode offers users the ability to configure both minimum and maximum numbers of event pollers for their SQS ESM. Each event poller acts as a unit of compute responsible for queue polling, event batching, and filtering before initiating Lambda functions. These pollers can handle significant throughput—up to 1 MB per second—and support up to 10 concurrent invokes or 10 SQS polling API calls per second.
Setting a minimum number of event pollers ensures that applications can maintain a baseline processing capacity, ready to manage sudden traffic increases without delay. It’s advisable to configure the minimum pollers to align with known peak workload requirements. Additionally, the optional maximum setting helps prevent overloading downstream systems by capping the total processing throughput.
The new provisioned mode significantly enhances the ability of event-driven applications to manage varying workloads. As traffic rises, the ESM detects any backlog within seconds and dynamically scales event pollers between the configured minimum and maximum values. This scaling occurs three times faster than before, complemented by a dramatic increase in processing capacity—able to handle up to 2 GBps of aggregate traffic and 20K concurrent requests. By maintaining a minimum number of ready pollers, applications achieve predictable performance, effectively managing sudden traffic spikes without the usual delays associated with scaling up resources. During periods of low traffic, the ESM automatically scales down to the minimum configured pollers, optimizing costs while maintaining responsiveness.
How to Get Started
Enabling provisioned mode is a straightforward process in the AWS Management Console. Users need to have pre-configured an SQS queue and a Lambda function. To start, navigate to the Configuration tab of the Lambda function, select Triggers, and then Add trigger. This opens an interface where you can set up your trigger. Select SQS from the dropdown menu as the source and choose the desired SQS queue.
In the Event poller configuration section, a new option called Provisioned mode will appear. Select Configure to access settings for Minimum event pollers and Maximum event pollers, with default and permissible values displayed.
Once you’ve set up the provisioned mode, you can save your trigger. Should there be a need for adjustments later, the current configuration can be accessed and modified under the Triggers tab in the AWS Lambda configuration section.
Monitoring and Observability
Provisioned mode usage can be monitored through Amazon CloudWatch metrics. The ProvisionedPollers metric provides insights into the number of active event pollers processing events in one-minute intervals. This level of observability ensures that users can keep track of their application’s performance and make data-driven decisions to optimize their event-driven architectures.
Availability and Pricing
The provisioned mode for Lambda SQS ESM is now available in all commercial AWS Regions. Users can take advantage of this feature via the AWS Management Console, AWS Command Line Interface (CLI), or AWS Software Development Kits (SDKs). Pricing for this feature is determined by the number of event pollers provisioned and the duration of their provision, measured in Event Poller Units (EPUs). Each EPU supports up to 1 MB per second throughput capacity per event poller, with a minimum of two pollers per ESM. For detailed pricing information, users are encouraged to visit the AWS pricing page.
Conclusion
The introduction of provisioned mode for SQS ESM marks a significant advancement in AWS’s offering for event-driven applications. With enhanced control and performance, users across various industries can build more responsive applications that meet their performance requirements while optimizing costs. For more information on how to implement provisioned mode for SQS ESM, interested parties can refer to the AWS Lambda documentation. This feature not only enhances the efficiency of event-driven architectures but also empowers organizations to maintain robust and scalable applications in the cloud environment.
For more Information, Refer to this article.
