Replaying Events using AWS
Background
Several processes as part of the Sirius Finance Hub are event driven. If any of these events fail to process correctly (shown in the dead letter queue), it is possible to replay them by sending the event again to the appropriate EventBridge bus.
Replaying an Event
This process is entirely done through the AWS Management Console. The failed event will be shown in the dead letter queue (DLQ) for the relevant environment and workspace (e.g. supervision production: events-dlq-production-supervision).
To view the DLQ click on Amazon SQS in the AWS Console, and search for the queue name. In the “Monitoring” tab you can see how many messages are in the dead letter queue, and roughly the time they occurred, which we can use the specify the replay time frame.
To replay the events on a particular bus, navigate to Amazon EventBridge in the AWS Console, select “Event buses” from the left hand menu, and search for the appropriate bus (e.g. production-supervision). Navigate to the “Archives” tab, select the archive for this bus and click “Replay” which will open the “Start new replay” page.
On this page, give your replay and name and description so others know why it has occurred. By default, under “Specify rules”, “All rules” will be selected, which will replay all rules for the bus. If you know the event type that needs to be replayed, you can create a new rule to match only that exact event type. Then specify the time frame for which this replay will occur. Here use the time frame you identified earlier in the monitoring tab of the DLQ, and press “Start Replay”.
This will replay all events on the bus in this time frame that match the selected rules (if any), including those that were successful. Each service that receives & processes events should process them in a way that avoids processing duplicates, but you should still aim to be as specific as possible with the replay criteria.