AWS Solutions for Event-Driven Architecture

January 25, 2023

Damian Borowiecki

Presentation of event-driven architecture

In the simplest terms, event-driven architecture is an approach to designing systems that produce and respond to events. Events can be defined as changes in state that initiate action in the system. While in request-response architecture, which is another common approach to building cloud applications, communication between components is achieved through API calls in a synchronous manner, the event-driven approach is asynchronous. Client issues an event and moves to performing another task. While request-response architecture is simpler, as the system grows in complexity, it becomes harder to manage further development. On the other hand, since event-driven systems are modular and decoupled, they can be more resilient to failures or errors in individual components. If one part of the system goes down, the rest of the system can continue to operate.

Event-driven vs request-response

The following example shows differences between event-driven and request-response architectures based on a home security system that sends a phone notification when the door is opened. In request-response architecture this could be presented as:

The Lock Service receives information that the door has been opened and then requests Notification Service to send the appropriate information to the mobile device. After that’s done, the Notification Service sends a success/failure response to the Lock Service.

In event-based architecture, the Lock Service responsible for processing the door signal does not call Notification Service directly, but instead it sends an event to the Event Bus (central hub that allows different components of the system to publish and subscribe to events). Event bus then passes the event to the Notification Service.

At first glance, the difference between the two solutions is not substantial, but it is worth considering how the chosen approach affects the further life of the application. Let’s assume that we want to add a function to the system that will additionally use artificial intelligence to identify the person who opens the door. If the system does not recognise the person, an alert is sent. In this case, the event of the door being opened would trigger two reactions: sending an alert to the phone and triggering an identity confirmation orchestration.

With every added functionality, responsibility entrusted to the Lock Service grows. It has to know what APIs need to be called in Notification Service and Identification Service. When other services are updated, it might be also necessary to update the Lock Service to keep it compatible.

Over time, it will become progressively more challenging to react quickly to changes. Components of the application are tightly coupled and while it might not be a huge issue when there’s full control over the system, decoupling is important when there are parts of the systems we have no control over. Event-driven architecture makes us less reliant on these external components.

In an event-driven approach adding new functionality to the system doesn’t increase the complexity of the Lock Service. New subscriber is added instead.

Differences in error handling

Let’s assume that an error in the identification process should result in a notification being sent to the phone. It would be necessary to add a notification service API call in the identification process or to synchronously call first identification and then notification directly in the Lock Service. Synchronous queries, although simple, can be prone to throttling problems. If many clients request notifications at the same time, it can cause significant system slowdowns. If the notification system goes down, it’d be necessary to wait for it to become available again before we can move on in Lock Service. In event-driven architecture, Identification Service would send separate events that will be passed to Notification Service. Events are transmitted to the hub via messaging services, these serve as buffers that can adapt to changes in traffic. As a result, the Event Bus will not be overworked. An example of an AWS service that performs such a role is SQS. It can scale almost infinitely. Consequently, scaling systems based on an event-based architecture is simpler.

AWS Support for Event-Driven Architecture

AWS approach to building applications is focused on business value rather than underlying technology. Provided services are made to simplify the use of the cloud, building solutions ‘in’ the cloud instead of ‘on’ it. By adopting a serverless approach and outsourcing some of the operations to the provider, it’s easier to create robust solutions for modern world challenges.

AWS provides a number of solutions for the smooth introduction and management of event-based architecture. Services that are useful in handling asynchronous communication and, therefore, in event-driven application are:

Amazon SQS – A message querying service. It allows secure communication between different components without requiring services to be available. Used to decouple and scale applications as well as to queue tasks that will be processed by other services.
Amazon EventBridge – Event Bus service for event routing. It allows receiving events from multiple sources and sends it to multiple targets. Used to choreograph events.
Possible targets:
- other AWS services,
- other buses,
- API destinations.
Possible sources:
- other AWS services,
- custom applications provided by the user,
- SaaS applications.
Amazon Step Functions – It allows creating workflows for decoupled applications as well as developing and visualizing event-driven architecture. Used to orchestrate processes.

Differences in calling Lambda functions synchronously (request-response architecture) and asynchronously (event-driven architecture)

In the first example Lambda function is called through an API call. Through Lambda function URL code is executed and the result returned is response. In the second case EventBridge sends events to the relevant subscriber’s queue. Queue sends back information that it received and then passes it to Lambda function.

Difference between event orchestration and choreographing events

In choreography events are exchanged using subscriptions. It is decentralized – different services are responsible for sending and receiving events independently from the choreograph (Event Bus). On the graphic below you can see that event sources are providing events, which are later on sent to the designated targets.

Orchestration coordinates flow of tasks executed within a domain and results in publishing an event. AWS Step Functions are orchestration tools that are responsible for creating and maintaining state machines. Each step in this process is called state, steps may include math operations, data manipulation, etc. Moving between states is called state transition.

Structure of event message

Copy Code

{
  "version": "0",
  "detail-type": "Door Status",
  "source": "SmartDoor",
  "time": "2022-12-121T12:00:00Z",
  "detail": {
    “metadata”: { “idempotency-key”: “6a5c14d7-f1ed-4e5e-9c76-44a3f718ed96” },
    "state": "open"
  }
}

JSON presents exemple event content for a signal sent by Smart Door. It sends a signal to the Event Bus, informing that the state of the door has changed. Events provide information about what happened in the past and are, therefore, immutable. Events don’t have to provide full information about the state, just information necessary from the point of view of the event subscriber.

On the other hand, a balance has to be found so that the amount of data provided is not too small. Example:

A smart door is located in an office building; when the door is opened, information about who is entering is also registered. We have access to full information about the person: first name, surname, role in the organization, etc.

Instead of sending all this information in the event detail directly, user reference can be provided. The ID of the person alone may not be enough; in addition, we may need to know where the information is stored. Event content should provide useful information without having too much information. The size of events is important for 2 reasons:

Backward compatibility – Events should be backward compatible. The more data we provide, the more difficult it is to change the content of detail while maintaining data consistency.
Cost efficiency – The more content is communicated directly through an event, the more likely we are to see costs increase over time.

Idempotency-key added to the details metadata is an ID that helps avoid duplication of operations. When data is sent to the receiver in asynchronous manner, but there’s a failure during acknowledgement – sender is unaware of previous data processing status and might want to retry operation. Therefore, we may find ourselves in a situation where an event concerning the opening of a door has been sent twice, but doors were opened only once. Idempotency-key is a unique identifier of operation provided by the client. By including it in event data – targets can check if an operation for given action took place in the past.

In order for an event target to receive information about an event, it is necessary to create a rule that matches events. EventBridge pattern corresponding to door open event might look like:

Copy Code

{
  "source": [ "SmartDoor" ],
  "detail-type": [ "Lambda Instance State-change Notification" ],
  "detail": {
	"state": [ "open" ],
“metadata”: {
“idempotency-key”: “6a5c14d7-f1ed-4e5e-9c76-44a3f718ed96”
}
  }
}

EventBridge Pattern rules:

Event must contain all fields names from pattern.
Fields not mentioned in pattern are ignored.
Matching is exact.
Number matching is at string representation level.

Summary

Event-driven architecture is an approach for creating systems that are decoupled, scalable and resilient. While it might be more complex in understanding that request-response architecture as dependencies are harder to understand, there are more moving pieces and additional aspects such as duplication of events that must be taken into account, it is a reliable way to develop next gen applications in serverless environments. AWS provides plenty of tools to facilitate the creation of solutions in this convention. In particular EventBridge, which allows the management of Event Buses, and Step Functions, which allow the orchestration of events and the visualization of this process.