What Duplicate Users Taught Me About Outbox and Idempotency

After enough support tickets involving students who couldn’t log in, I stopped seeing each case as an isolated data problem. Staff operations were failing too, and I kept returning to the same boundary between two services.

For this post, I’ll call those services Student and Users. The names are simplified, but the requirement is one we had at work: creating a student in Student also required an application-user record in Users.

On paper, the flow was short. Student asked Users to create the user, received a userId, and then saved the student. The difficult part was everything the network could leave unanswered between those two local transactions. Student might not receive a response even though Users committed its work. Retrying could recover the operation, or repeat an effect that had already happened.

Transactional outbox, idempotency, and inbox each address part of that problem. The outbox preserves the work Student still needs done. An idempotency key gives repeated attempts the same identity. The inbox remembers what Users already did and what it returned. I’ll add them in that order, starting with why the direct call seemed reasonable.

I first worked on this path when I was a junior. We knew that one service’s transaction did not magically include the other service. The direct call was a conscious tradeoff for a one-to-one interaction, not an attempt to pretend the boundary did not exist.

Its appeal was easy to see on the happy path. Student called Users, took the returned ID, and finished saving the student. The code could read almost like the requirement.

The support cases made the limit harder to ignore. One student’s login reached the application-user lookup and failed there. When we inspected the data, we found three enabled user records sharing the same document number. Other cases involved students who couldn’t log in or staff operations that failed because the expected student/user relationship was missing or inconsistent.

The symptoms varied, but I kept ending up at the same boundary. Student expected one usable Users record for the student. When that relationship was missing or ambiguous, an operation that looked like a student problem could fail in Users instead.

My reaction was not particularly sophisticated: this is absurd. I knew I could control the requests to Users better, yet I was still spending time repairing the consequences when that boundary went wrong.

I still do not know which historical operation created those three records. A repeated request is possible, but other sequences are possible too. I can explain the failure at the service boundary without pretending that it proves the cause of that incident.

The direct call still contains two transactions#

At the business level, creating a student and its application user sounds like one action. The state is still split. Student owns the student record, while Users owns the user record.

User creation is part of student creation. Student needs the returned userId to connect the records and finish its work. Losing that result leaves the business operation unfinished from Student’s perspective, even if Users has already done its part.

Each service can make its own changes atomic. What the call cannot do is merge those local transactions into a single commit or rollback across both services.¹ Once Users commits, Student cannot undo that transaction as if both changes belonged to one database.

The direct request also makes Student wait for Users. A failure before Users commits can stop student creation. A failure after Users commits is more awkward because the user may exist even though Student did not finish its side.

Suppose Users creates the user and returns a userId, but the response never reaches Student. Student knows that the answer is missing. It does not know whether Users completed the request. Sending the request again may recover the operation, or it may repeat the effect unless Users can recognize the attempt.²

A timeout says that the answer did not arrive. It does not say what Users committed.

The outbox preserves Student’s intent#

I first reached for an outbox after dealing with notifications that had not been delivered. What bothered me was not only that they failed. Once the original request was gone, we had little durable state to help us send them again.

While designing that flow, I started looking at other service boundaries where the same idea could help. Student and Users was one of them.

Here, the problem was no longer only that the call could fail. Student’s need to create a user lived inside that one request, and once the request was gone, Student had no durable record it could return to.

An outbox gives Student that record. Student saves its own change and a create-user intent in the same local transaction. If the transaction commits, both are stored. If it rolls back, neither is stored. A separate sender can process the intent afterward.¹

That was the part I had been missing on the Student side. The request could end without taking the required work with it. The outbox row would still say which operation needed to happen and which student it belonged to.

But keeping the intent available also means a sender may deliver it more than once. Suppose Users creates the user and returns the result, but Student does not record that success. The outbox row still looks unfinished, so sending it again is reasonable from Student’s side. Users, meanwhile, may have completed the work already.¹

Student now has a way to recover the intent. Users needs a way to recognize that the next delivery belongs to the same operation.

Repeated delivery introduces idempotency#

This is where idempotency enters the design. Student assigns one idempotencyKey to the create-user operation and reuses it every time the outbox sender delivers that operation. A fresh key on each attempt would make those attempts look like separate operations to Users.²

Student can still retry. The key gives Users a stable identity for understanding that those attempts belong to the same piece of work.

The terminology can make this sound more abstract than the Student-side code needs to be. The TypeScript examples in this post are invented, incomplete, and use Prisma-style calls only to illustrate the two local transactions. They are not production code.³

The Student transaction creates the student and its outbox row together:

async function createStudent(studentData: StudentData, idempotencyKey: string) {
  return prisma.$transaction(async (tx) => {
    const student = await tx.student.create({ data: studentData });

    await tx.outbox.create({
      data: {
        idempotencyKey,
        studentId: student.id,
        operation: 'create-user',
      },
    });

    return student;
  });
}

The outbox row keeps the same idempotencyKey whenever the sender tries to deliver this create-user operation again.

The inbox keeps the Users result#

Once Student can retry, Users needs more than a way to say, “I have seen this before.” Student still needs the userId from the first attempt.

If Users stores only the key, it can avoid creating another user but cannot finish the original conversation. The inbox needs to keep both the idempotencyKey and the result that belongs to it. A repeated request can then receive the same answer instead of triggering the same change again.⁴

Here, the inbox is an ordinary table. When the key is new, Users creates the user and the inbox row in one local transaction. When the key already exists, Users returns the result it stored earlier.

Keeping both writes in the same transaction is what makes the record trustworthy. If user creation fails, there should be no inbox entry claiming it succeeded. If the inbox write fails, the user creation rolls back with it.

The Users-side code can stay fairly small:

async function createUser(userData: UserData, idempotencyKey: string) {
  return prisma.$transaction(async (tx) => {
    // Ensure it is processed once.
    const previous = await tx.inbox.findUnique({
      where: { idempotencyKey },
    });

    if (previous) return previous.result;

    const user = await tx.user.create({ data: userData });
    const result = { userId: user.id };

    await tx.inbox.create({
      data: { idempotencyKey, result },
    });

    return result;
  });
}

The example assumes idempotencyKey is unique. Two concurrent calls might both check the inbox before either one inserts the row, but only one can claim that key. The other transaction rolls back and can retry. On its next attempt, findUnique returns the result already stored for the operation.³

Put the two local transactions side by side and the change becomes easier to see:

Current synchronous flow
Student -> Users: create user
Users -> Users state: save user
Users --> Student: userId
Student -> Student state: save student

Outbox and inbox flow
Student -> Student state: save student + outbox (local transaction)
Outbox sender -> Users: create user [idempotencyKey]
Users -> Users state: save user + inbox result (local transaction)
Users --> Outbox sender: userId
Outbox sender -> Student state: associate userId
Outbox sender -> Users: repeat create user [same idempotencyKey]
Users --> Outbox sender: stored userId

This flow trades immediate agreement for recoverability. Student can commit its record and outbox entry before Users creates the user, so the relationship may remain incomplete for a while. The outbox does not decide what the application should expose during that window; that is a separate product decision.

Idempotency does not replace business rules#

The same idempotencyKey tells Users that this is another attempt at the same operation. A different key identifies a different operation, even when some fields in the two requests happen to match.²

That is all the inbox should decide. It should not compare different requests and silently merge them because their payloads look similar. Whether two separate operations are both allowed belongs to domain validation and uniqueness rules, not to the inbox.

Repeated delivery, one business effect#

The outbox may send the same operation more than once. For one idempotencyKey, Users can recognize the repeat, skip the second user creation, and return the userId it saved the first time.

The delivery still happened more than once. The create-user effect did not. That is the scoped guarantee here, rather than literal exactly-once delivery.⁵

What I wanted from this design was not a magical transaction across two services. I wanted each service to know what it owned once the easy request path stopped being easy. Student keeps the work it still needs done. Users keeps the result of the operation it already processed.

I first built the direct path as a junior. Later outbox work gave me a better way to reason about it, so I came back to the same boundary with different questions. That is a separate article I want to write: how revisiting earlier work has become part of how I learn.