Skip to content
derpx06Writing / Distributed Systems
0% · 5 min leftSubscribe
Distributed Systems · October 8, 2026

Mobile Apps Are Distributed Systems With Terrible Networks

A phone is a copy of your data that vanishes without warning, holds changes for hours, and comes back disagreeing with everyone else. Most mobile bugs are really that problem, unlabelled.

A salesperson opens a property app in a basement showroom. They update three deals, add a note, and put the phone back in their pocket. Ninety minutes later they come up for air, the app reconnects, and everything they did has to be reconciled with a server that has meanwhile accepted changes from two colleagues and an overnight import.

Nothing in that paragraph is really a mobile problem. It is the problem of keeping several copies of the same data in agreement when they cannot always talk to each other — the exact thing that databases spread across data centres are built to handle.

The only unusual part is that one of the copies is in somebody's pocket, and the loss of connection is called "a lift."

We do not normally describe mobile work this way. I think that is precisely why so much of it goes wrong.

Server engineers work with networks that are fast, reliable, and fail loudly. Phone networks are none of those, and the third one does the damage.

Your "are we connected?" check says yes on a train that has a signal and no actual throughput. A request succeeds and its reply never arrives, because the radio went to sleep. The phone freezes your app and thaws it later — from the app's point of view, instantly — but four minutes have passed and the login has expired. And an airport WiFi hotspot cheerfully answers every request with a login page, so your code that expected data is now handling HTML.

Each of these produces a report that says "sometimes it doesn't save," which nobody can reproduce at a desk on office WiFi.

They are not flaky. They are the correct behaviour of a system whose assumptions do not match where it is running.

The usual first attempt is to keep a copy of the last answer and show it when the network is down. This makes the app feel nicer and changes nothing underneath, because saving still assumes the server is reachable.

Offline-first means something stronger: the copy on the phone is the truth as far as the screen is concerned, and talking to the server is a background chore that catches up later — possibly much later, possibly after the app has been closed and reopened.

That flip has consequences you have to design for rather than discover.

The phone must name things itself. If the server assigns the ID, then something created offline has no ID — and anything referring to it, like a note attached to a brand new contact, cannot even be expressed. Generate a unique ID on the device at the moment of creation and let the server accept it. This one decision deletes an entire family of sync bugs.

Every record needs a status. Not a yes-or-no flag. At minimum: waiting, sending, saved, failed. The screen has to be able to show a row as not yet on the server, because that is the honest thing to tell somebody — your change is safe on this phone and has not reached us yet. Hiding that distinction is how you get a person who believes their work is saved when it is stuck behind a permanent error.

The waiting list must survive the app being killed. The phone's operating system will shut your app down mid-sync. If the list of pending changes lives only in memory, that work is gone and the person was told it was fine. It belongs in durable storage — ideally saved in the very same step that saved the data.

Two people edited the same deal. One of them was offline. Something has to give.

The engineering world offers a menu of answers: last one wins, merge automatically, keep both, let the server decide. Picking from that menu first is the mistake.

The right question is what a person expects to happen — and the answer changes field by field within a single record.

A deal's stage is a position in a process, so the most recent deliberate change should win, and a stale offline update quietly undoing a colleague's progress is a genuine bug.

The notes field is append-only in spirit. Two people adding notes should produce both notes. "Last one wins" silently destroys one of them.

The owner field probably should not merge at all. It should raise its hand and ask, because two people each believing they own the same account is a real-world problem no algorithm can settle.

So: decide per field, based on what that field means. Last-one-wins for genuinely single values where the stakes are low. Append for anything that behaves like a log. Ask a human for the small set where quietly choosing a winner is worse than pausing.

A phone retries constantly. The operating system retries. The network layer retries. Your own sync queue retries after a crash. Which means the server will receive the same change more than once as a matter of routine, not as a rare edge case.

If creating a deal is not safe to repeat, that basement showroom produces three deals.

The fix is the one from anywhere else: the phone generates a unique ID per logical change, the server records it with the result, and a repeat gets handed back the original outcome instead of doing the work again. On mobile this is not a nicety for rare failures. It is load-bearing, because duplicate delivery is the expected case.

The device-generated ID from earlier does double duty here, which is a pleasant accident of good design.

Once you accept that a phone is just another copy of the data, the mobile bug reports stop looking like a random assortment and start looking like a short list of known problems with known answers.

Sometimes it doesn't save is a durable-queue problem. It saved and then went back is a disagreement-resolution problem. It made two is a repeat-delivery problem. It showed me old data is a cache problem.

None of those is fixed by a better network library or a stricter connectivity check. They are fixed by admitting you are running a system with several copies of the truth, one of which is unreliable, occasionally lies about the time, and is being carried into a basement — and then applying the ordinary, well-understood answers to that situation.

The network is not going to get better. Your model of what you are building can.

Keep readingYour Retry Made It Worse5 min · Distributed Systems

Related reading

Distributed Systems · 5 min

Your Retry Made It Worse

Trying again after a failure is the most obvious safety net there is, and the easiest one to build backwards. It hinges on one question: did the first attempt actually fail?

The monthly letter
One email a month

What I read, built and got wrong.