The two dashboards will disagree

Sales shows 412 orders. Finance shows 387. The founder app and the investor portal disagree on last night’s GMV. The till printed one amount. The customer’s MoMo SMS printed another. Decide this week whether those two surfaces must show the same number at the same second, or a bounded lag is acceptable.

Two readers of one ordered stream sit at different offsets. While writes continue they lack timeliness with respect to one another. That is a design choice until a user-visible join exists. Then it is a defect.

Why these three models

The decision is whether two live surfaces must agree this second. The features: an unread offset that accumulates, two processors of one stream that keep moving, and a second action that either waits or fires into a stale view.

Three lenses, three outcome types. Timeliness-lag is a cycle answer: two clocks on one log disagree for a duration, then agree, for as long as writes continue. Stocks and flows give a complex answer: the unread offset is a level you move only through its flows. Signal-threshold gives a random answer: the cutoff that blocks the second action is set by miss cost, false-alarm cost, and how often a join happens while the offset is still open. Cycle, complex, random. That span is the point.

LOOP is folded. The customer and the investor will be back next period, but the lever is the join, not a patience inequality. Behavioral is folded too: people treat the disagreement as a bug and call a meeting. The meeting does not move either offset. Governance, who may declare a join serialized, sits in the dated portfolio. Neither added a lever of its own.

The framework: two clocks on one stream

1. The two positions: they lack timeliness while writes continue

Ben Stopford’s worked case is a confirmation email and an orders view. Both subscribe to the same Validated Orders stream and process it concurrently. One will lag behind the other. If a later business operation consults both, the user sees an inconsistency.¹

He then names the condition that closes it. Stop writes and both the orders view and the email service eventually converge on the same state. In ordinary operation they sit at slightly different positions in the event stream, so they lack timeliness with respect to one another. The customer clicks the link in the confirmation email. The view has not caught up. The link fails, or it returns the wrong state.²

Those are his words: lag behind, lack timeliness, different positions in the event stream. Do not cite him for a Kafka ops term he does not use.

A WhatsApp order hits the sales phone. The stock sheet is a second reader of that day. A MoMo receipt lands on the customer’s phone. The till is a second reader of that payment. Two teams counting revenue at order and at cash are answering two questions that share a name. Here both surfaces are answering the same question from the same stream, from two offsets.

If nobody consults both, the disagreement is invisible. You bought concurrency and you pay in lag. That bargain is legal. The moment a person can see both, or can act on one by using the other, you have a join: the email link, the investor who opens the portal after your update, the customer holding a MoMo SMS against a till slip.

The book’s repairs are specific. Block the second action until the view updates. Or have the orders service raise a View Updated event, and let that event, not the original one, trigger the email. Both put serial execution back in where the join exists, and leave the rest of the system concurrent.²

The people running the two surfaces are usually trapped. The email fires because an order event arrived. The till prints because a button was pressed. A lecture about being on the same page treats a structural offset as a character flaw. Change whether the second action is allowed to fire, or stop pretending the two screens are one clock.

Timeliness-lag, the cycle lens

• Assumes: two processors read one ordered stream independently. Writes continue. No shared clock gates either answer.
• Fits because: two live surfaces of one fact stream, and writes have not stopped.
• Breaks when: writes stop and both catch up; the designer serializes the join; there is only one reader; nobody can see both.
• Evidence: grade B. Practitioner (Stopford 2018). Bound: no RCT. Worked failure is email-before-view.
• Counteracts: treating every disagreement as a bug, or as a definition fight.
• May reinforce: accepting a visible join as ordinary weather.

2. The unread offset is a stock

Unread offset is a stock. Events written, minus events the slower reader has applied. You cannot set it. You can raise the consume rate, cut the write rate, or stop writes. A meeting does none of those.

The water in a tub is filled by the inflow and drained by the outflow. No feedbacks required. The level moves only through those two rates.³ Your disagreement is the water. Hiring a data person drains it only if that person raises the slower reader’s rate or cuts what is being written. Stopping the shop drains it, which is why “it matched this morning before we opened” is not evidence the systems agree. You closed the tap.

Measure the stock. Count the events the slower surface has not yet applied. That count, divided by that surface’s processing rate, is the disagreement window you are actually running. Werner Vogels calls the same interval an inconsistency window and notes that it can be sized from delays and load, provided no new updates arrive.⁴ While writes continue, the stock has a floor set by the gap in the two rates.

Queueing prices wait time. The harm here is disagreement at the moment both surfaces are consulted. A customer who waited thirty seconds for a till that then matched the MoMo SMS had a wait. A customer who waited zero seconds and received two different amounts had a defect.

Overwrite is a different week’s decision. The log can be honest and append-only. Both readers can be faithful. They will still disagree, because they are at different positions of a truthful stream.

Draw the boundary. Inside: your own readers of one ordered fact stream, whether writes continue, and whether a person can see both. Outside: the mobile-money provider’s settlement file, the bank’s ledger, WhatsApp’s message store. You do not own those offsets. If MTN’s SMS and your till disagree, one of the two clocks is not yours. Treat that as a join with a counterparty, not as a bug in the till.

Stock-flow, the complex lens

• Assumes: unread offset is a level. It changes only through write rate and consume rate.
• Fits because: the disagreement lasts, then closes. That is a stock emptying.
• Breaks when: the two numbers are different definitions, not different offsets, or the quantity is not conserved.
• Evidence: grade A for conservation. Sterman on stocks and flows.
• Counteracts: trying to set the disagreement to zero in a meeting.
• May reinforce: stopping the shop to drain the tub and calling that a fix.

3. The cutoff: when the disagreement is large enough to block the next action

Not every disagreement should stop the second action. The cutoff is arithmetic on three numbers you already have, even if nobody wrote them down.

The miss: the second action fires while the first reader is still behind. Dead confirmation link. Portal that understates last night’s collections. Till that has not seen the MoMo credit the customer is holding out.

The false alarm: you blocked a second action that would have been fine. Email delayed an hour. Till waiting on a view that had already caught up.

John Swets’s result, from diagnostic systems rather than dashboards, is the same dial. Any imperfect detector produces hits and false alarms. The right decision criterion depends on how rare the event is, and on the costs of the two errors.⁵ Accuracy lives on a curve, and you pick a point on it.

Money and identity joins sit at one end. A miss can take the customer’s trust or the investor’s reading of the round. Keep that cutoff sensitive. Make the false alarm cheap: a short block, or a View Updated event that releases the second action once the first reader is ready.

Internal sales and finance screens with no customer in the room sit at the other end. A miss is an awkward Monday. A false alarm that stops invoicing is a real cost. Accept the lag. Label each screen with the as-of position it has reached.

The prior is the number founders skip. If a join is rare, a loose cutoff still produces few visible defects. If every customer holds a MoMo SMS against the till, the same offset produces a defect on every sale. Count how often someone can see both surfaces in the same minute. That count sets how tight the cutoff has to be.

Signal-threshold, the random lens

• Assumes: join-while-behind and join-after-catch-up have estimable costs. The prior on a live join can be counted.
• Fits because: every second action is a cutoff on whether the first reader is ready.
• Breaks when: the base rate of joins drifts and the cutoff goes stale, or there is no shared stream at all.
• Evidence: grade A. Settled detection theory (Swets).
• Counteracts: serializing every screen, or serializing none.
• May reinforce: a precise cutoff on two numbers that were never the same fact.

GEER: how to close a join without stopping the shop

Exposure sits on the joins. A sales screen thirty events behind finance, with nobody looking at both, is a stock you can live with. A confirmation that goes out before the account page has the order is a join you have to close. Ruin sits on money and identity: a MoMo-versus-till mismatch the customer can photograph, a portal that tells an investor a different committed figure than the update you just sent. Cap any change so it cannot drop a payment or a KYC event while you are aligning the views.

Cheapest and most reversible first.

1. Name the two surfaces and the stream they claim to share. If they do not share a stream, stop. You have two facts, not two offsets. An afternoon.
2. Name the joins. Where can a person see both, or act on one by using the other. Write them as a list. An hour.
3. Count the unread offset on the slower surface. Events in, events applied. That is the stock. A morning.
4. Write a lag budget per join. How many events, or how many seconds, the second action is allowed to see. Free.
5. Serialize money and identity joins. Block until the view updates, or fire the second action off a View Updated event. A day to wire. Use the book’s repair only where the join can move cash or a name.
6. Label the rest. Internal screens get an as-of. “Finance, through event 14,802, 11:04.” The meeting then argues about a stock, not about who is lying.
7. Do not add a third dashboard as the referee. A third reader is a third offset. It does not close the first two.

Count the states the pair can be in, then count the responses you actually have. If the only response is discuss it on Monday, you cannot regulate this. The list above adds response categories. Headcount does not.

RADAR: what to lock before the next click-through

Do now, by T+3, one afternoon, effect visible the same day. Write the list of joins. Mark each one money, identity, or internal. Reversible: you can reclassify. The effect arrives the next time someone is about to send a confirmation, publish a portal number, or promise a crate off the WhatsApp thread. The list does not pick a winner. It picks the places a person can see both.

Hedge, by T+14, premium is a short block on the dear joins, cover live before the next customer or investor looks at both. Put serial execution on money and identity only: the till waits for the payment view, or the confirmation waits for View Updated. The premium is latency on those actions. The cover has to be live before the next dropped-connection Saturday, or the hedge is a note.

Defer and trigger, size declared when the trigger is set. A company-wide sync layer, or a halt-the-writes ritual at every close, is expensive and mostly irreversible in habit. Pre-commit the observable: two money-or-identity joins in one week that the lag budget failed, after the hedge is live. When that fires, you may buy the heavier machinery. The trigger is a person who can see both being shown two numbers you had already decided must match.

The list of joins has to exist before you can see that trigger.

CHAIN: companies that run two live numbers

Name the shape before you explain it. This is two clocks on one log. The disagreement opens when writes resume, holds while the rates differ, and closes when the slower reader catches up or the writes stop. It is not oscillation. Nothing overshoots. It is not a queue whose harm is waiting. The signature is a pair of numbers that later agree, having disagreed in public.

Ask the break questions before you borrow anyone else’s history. Has a rule changed, has an actor entered or left, has a measurement become a target? The last one fires often. If sales is paid on the sales dashboard and finance is judged on the finance one, the offset is now an incentive instrument. Last year’s “they usually catch up by noon” then describes a different process. The lag budget has to be tighter, because somebody is now rewarded for their own clock.

Stand this next to any shop that lets two processors read one ordered fact stream and then lets a person consult both: confirmation before the account page, WhatsApp thread against the stock sheet, till against the MoMo rail. Across that class the directional base rate is blunt. While writes continue, the two readers disagree. Teams that treat the disagreement as surprising spend the week on blame. Teams that named the joins spend it on the cutoff.

Strip the lookalikes. Two correct numbers for two definitions are a different class. A cell that was overwritten is a different class. If the definitions match, the stream is shared, and the numbers still differ, you are looking at offset. Credit the catch-up only if the consume rate changed or the writes stopped. A quiet afternoon is not a process improvement.

One flag. If a regulator, a bank, or a lead investor requires the two surfaces to match at a named timestamp, the join has been declared for you. Serialize that one. Do not generalize the halt to every internal screen.

What these three cannot time

These models can tell you that two readers will disagree while writes continue, that the unread offset is a stock, and that the cutoff belongs on the join. None of them can tell you how many seconds your pair will stay apart when MTN is slow, or whether this investor will read a ninety-second gap as sloppiness or as fraud. You have agreed to be surprised by both.

A second limit cuts against the whole argument. This applies when the two surfaces really do share one ordered stream. Two numbers that were never the same fact will not be fixed by a View Updated event. Applying this piece to a definitional fight would serialize the wrong thing.

One property no member models: once you serialize a join, people start treating every internal screen as if it must also wait. The habit of promising a crate off the WhatsApp thread, then updating the sheet later, was absorbing awkward cases. That absorption does not appear in the offset stock. You will notice it only when it stops.

The one action that survives the ignorance: before Friday, write the list of joins. Mark each one money, identity, or internal. Put a lag budget on the first two. Leave the internal screens labelled, not frozen. The decision that remains is which second actions are allowed to fire while the first reader is still behind.

Who names the join this week

The person who needs this is usually the one about to send the confirmation, publish the portal, or chair the meeting where sales and finance bring different printouts. The cheapest first test costs an afternoon: name the surfaces, name the joins, count the unread offset on the slower one. If a customer or an investor can already see both, serialize that join, or accept that they will do the detection for you, with a cutoff you did not set.

Sources and notes

1. Ben Stopford, Designing Event-Driven Systems: Concepts and Patterns for Streaming Services with Apache Kafka, O’Reilly, 2018, chapter 11, printed p. 103. Under the heading Timeliness: if two services process the same event stream they will process them at different rates, so one might lag behind the other; if a business operation consults both, this could lead to an inconsistency. Printed pages follow the book’s own pagination (printed = PDF page minus 15). Author landing for the free O’Reilly edition: benstopford.com.
2. Stopford, ibid., chapter 11, printed p. 104. The email service and the orders view subscribe to the same Validated Orders stream and process concurrently, so one will lag slightly behind the other. If writes stopped, both would eventually converge on the same state; in normal operation they sit at slightly different positions in the event stream and therefore lack timeliness with respect to one another. If the user clicks the confirmation link while the orders view is lagging, the link fails or returns an incorrect state. Repairs: the call to the orders service may block until the view is updated, or the orders service may raise a View Updated event used to trigger the email after the view has been updated. Both synthesize serial execution where it is necessary.
3. John D. Sterman, System Dynamics: Systems Thinking and Modeling for a Complex World, MIT Engineering Systems Division working paper ESD-WP-2003-01.13, 2002, section headed “Stocks and flows.” Stocks and flows are fundamental to the dynamics of complex systems. The stock of water in a bathtub is filled by the inflow and drained by the outflow, with no feedbacks, time delays, nonlinearities, or other complexities required. Open-access text layer: dspace.mit.edu. The conservation claim is the finding used here. The paper’s experimental failure rates are a different finding and are not quoted.
4. Werner Vogels, Eventually Consistent, Communications of the ACM 52(1), 2009, pages 40 to 44. Eventual consistency is a form of weak consistency: if no new updates are made to the object, eventually all accesses will return the last updated value. If no failures occur, the maximum size of the inconsistency window can be determined from communication delays, load, and the number of replicas. Author copy of the same text (the 2008 revision that CACM then published): allthingsdistributed.com/2008/12/eventually_consistent.html. Used here for the close condition (writes stop, readers converge) and for naming the disagreement duration, not as a second model.
5. John A. Swets, Measuring the accuracy of diagnostic systems, Science 240(4857), 3 June 1988, pages 1285 to 1293. Diagnostic systems distinguish signals from noise. Hits and false alarms are the two major proportions. Any operating system, unless perfect, will lead to false alarms as well as hits. The particular decision criterion depends on the prior probability of the event and on the benefits and costs ascribed to the correct and incorrect outcomes. Author-hosted JSTOR scan (text layer present): wixtedlab.ucsd.edu.

A note on what is deliberately absent. This piece does not tell you which cell may be overwritten. That is a memory question. It does not tell you what to do when two correct numbers answer two different questions. That is a definitions question. The only decision here is whether a second action may fire while the first reader of the same stream is still behind.

Joshua Agonya Pi’Rwot, Founder.