Brighter and the Outbox Pattern: At-Least-Once Delivery for Resilient

[Sascha]

Introduction


I'm going to start a Brighter series to talk about outbox pattern
In this article, we’ll explore the Outbox Pattern and how Brighter provides native support to implement it seamlessly. This pattern is critical for ensuring transactional consistency when publishing messages alongside database updates, especially in distributed systems where traditional two-phase commits (2PC) are not feasible.

Context


Before diving into the Outbox Pattern , let’s explore the problem it solves.

Problem 1: Single Message After Database Update


Imagine building an application using a messaging gateway. Initially, your workflow might involve updating a database entity and publishing a message (e.g., OrderCreated). However, if the message broker fails during publishing (e.g., network issues), the database update succeeds but the message is lost. This creates inconsistency between your system state and downstream services.

A naïve solution might involve retries.

Problem 2: Multiple Messages with Atomicity Requirements


As your application grows, you may need to publish multiple messages (e.g., OrderPaid and InventoryUpdated) within a single transaction. If the second message fails to publish, the first might already be delivered, violating business requirements for idempotent or atomic message delivery.

For example:

1. Update a database record (e.g., order.Status = "Paid").
2. Publish OrderPaid and InventoryUpdated.
3. If InventoryUpdated fails after OrderPaid is published, downstream systems receive an incomplete workflow.

This scenario highlights the need for transactional guarantees when combining database writes and message publishing.

Why Traditional Solutions Fall Short

• 
  Two-Phase Commit (2PC): Most databases and message brokers (e.g., Kafka, RabbitMQ) don’t support 2PC, and even if they did, it introduces tight coupling and performance overhead.
  
  
• 
  Manual Retries: Without transactional isolation, retries may publish duplicate messages or fail to recover from partial failures.
  

Outbox pattern


The Outbox Pattern ensures at-least-once message delivery while maintaining transactional consistency between database updates and message publishing. It guarantees that messages are only published if their corresponding database transaction succeeds, avoiding partial failures (e.g., database updates without message delivery).

How It Works


Instead of publishing messages directly to a messaging gateway (e.g., Kafka, RabbitMQ), the Outbox Pattern stores messages in a transactional database (e.g., PostgreSQL, MySQL, DynamoDB) within the same transaction as the business data. This ensures:

• Messages are persisted atomically with database changes.
• A background message relay process (e.g., a sweeper) reads messages from the outbox and publishes them to the messaging gateway.
• If the application fails before publishing, the message remains in the outbox and is retried when the system recovers.

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

Key Advantages

1. Transactional Consistency: Messages are stored in the same transaction as database updates, eliminating the need for distributed transactions.
2. Fault Tolerance: If the messaging gateway fails, messages are retried until acknowledged.
3. Decoupled Architecture: Applications focus on local transactions; message delivery is handled asynchronously.

Brighter's Implementation of the Outbox Pattern


Brighter provides native support for the Outbox Pattern across multiple providers (e.g., PostgreSQL, MySQL, DynamoDB).

How the Post Method Works Internally


When you use the Post method to send a message to a messaging gateway, Brighter implicitly leverages the Outbox Pattern under the hood:

1. Deposit:
   • The Post method internally calls Deposit, which stores the message in the configured outbox provider (e.g., InMemory, PostgreSQL) within the same transaction as your database operation.
   • This guarantees that messages are only persisted if the transaction commits successfully.
2. ClearOutbox:
   • After the transaction completes, ClearOutbox is invoked with the message ID.
   • This method publishes the message to the messaging gateway (e.g., RabbitMQ, Kafka) and deletes it from the outbox upon successful delivery.

Default Behaviour


By default, Brighter uses an in-memory outbox. While suitable for testing and for application that doesn't required guaranteed at least once, this provider lacks durability, as messages are lost if the application crashes before publishing.

How to configure it


Configurating it on Brighter will need to:


services
.AddBrigther()
.UseInMemoryOutbox() // or .UseExternalOutbox(....) for external provider
.UseOutboxSweeper(options =>
{
options.TimerInterval = 5;
options.MinimumMessageAge = 500;
});




And you will need

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

package for sweeper and

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.

for register Brighter in the Microsoft DI.

Conclusion


The Outbox Pattern is a foundation of reliable microservices communication. By leveraging Brighter's native support for this pattern, you can:

• Ensure transactional consistency without 2PC.
• Achieve at-least-once delivery with retries.
• Decouple database updates from message publishing.

In future articles, we'll explore Brighter's outbox providers (e.g., PostgreSQL, DynamoDB) and advanced configurations like message deduplication and custom sweeper logic.

Reference

Источник:

Пожалуйста Авторизируйтесь или Зарегистрируйтесь для просмотра скрытого текста.