The right position at the wrong time

A truck loses its signal at ten in the morning and gets it back at noon. As soon as it is back, it sends two things at once: the positions it stored with no signal and the ones it is generating right then. All of them are correct. If the platform does not tell them apart, it mixes history with the present.

A position can be technically correct and still lead to the wrong operational decision if it is read outside its time context. That is what this note is about.

We have worked for years with GPS tracking, telemetry and embedded devices, over GSM/GPRS cellular networks and over LoRaWAN. On the move, you cannot assume the connection will always be there. What matters is not only how long a device is offline, but how it behaves meanwhile and what happens when it comes back.

The figures in the examples are illustrative: they come from a simulator with an assumed working day of a fleet, not from measurements of our platforms.

It is not a device connected to a database

The simplest way to think about telemetry is a device that sends data and a database that stores it. With that picture, store-and-forward looks like a device problem: give it enough memory and have it send what it stored.

Our evolution has been to stop thinking about it that way. Today we see it as a distributed system where each stage has its own job: receive, acknowledge, transport, store, interpret and present. The problem does not end when the device manages to resend its data. What the platform does with it matters too.

On the device side, there is no general rule either. How it stores, how much it holds and what it does when it fills up depend on the manufacturer, the firmware and the model. Some store and send later; others have memory limits or settings of their own. It has to be checked device by device.

Three different times for the same data point

Every position has three moments:

• When it was generated, on the device.
• When the server received it.
• When it became available for operations, already processed.

On a day without failures, the three nearly coincide. After a disconnection, they drift apart: a position generated at 11:30 can arrive at 12:00 and, with the server’s queue full, become available up to four minutes later.

The device clock deserves attention too. With GPS, the device can take the time from the positioning system, but not every device keeps reliable time under every condition. And if the event time is not reliable, nothing built on top of it is either.

Acknowledging is not processing

When a device gets its signal back, there is no guarantee that everything arrives, in order and without duplicates. That depends on the protocol and on how it was implemented. In TCP integrations with protocols such as JT808, you need to understand their sequence, acknowledgment and retransmission mechanisms.

And there is a critical difference: the gateway acknowledging a message does not mean the whole platform has already processed and stored it. If an acknowledgment is lost, the device sends the same thing again, and the server has to accept the duplicate without creating duplicate records or events.

Recognizing a duplicate is not trivial either. Matching device and time is not enough. Neither is the sequence number: in JT808 it is two bytes and wraps back to zero, so it repeats. The right approach is idempotency per protocol and event type: a unique message identifier when there is one and it is reliable, and composite keys when needed.

Some protocols help from the moment of reception. In JT808, the batch position upload, message 0x0704, says whether the positions are normal ones or were stored in an area with no coverage. With that flag, the gateway can recognize history as soon as it arrives. The flag applies to the whole batch, and some devices send stored positions as normal ones, without it. Whether each device model and firmware version uses it is something to verify, not assume.

Accepting connections is not processing them

Receiving one position from each device at regular intervals is not the same as receiving, all at once, the backlog of many devices while their current positions keep coming in. Accepting TCP connections is one capacity; processing and storing everything received is another.

In the simulator, a fleet of 50 devices sends one position a minute for eight hours. At 10:00 the network goes down for 41 of them, for two hours. Each device holds up to 100 positions and, when full, drops the oldest. Once back, it sends the present without waiting and resends what it stored at two records a second, and 2 % of the acknowledgments are lost. The platform processes 10 messages a second, twelve times the fleet’s normal load.

The signal returns at 12:00, and in less than a minute the server’s queue reaches 3,634 messages; the wait, 6.1 minutes. In that queue, 63 current positions waited more than five minutes behind the fleet’s accumulated history. Along the way, 843 positions were lost because memory filled up, 436 duplicates arrived from lost acknowledgments, and 1,728 arrivals were out of order.

More memory does not fix everything. On the same day, with 500 positions per device nothing is lost, but the avalanche grows: the queue reaches 4,406 messages and 139 current positions wait more than five minutes.

History or present

The most delicate part is not only losing data. It is receiving valid data outside its time context.

If the platform treats everything that arrives as if it were happening now, history mixes with the present, and that hits routes, reports, geofences, alarms and indicators. With a geofence it is very clear: if a vehicle entered a restricted zone three hours ago and the position is only now reaching the server, an alert saying it has just entered makes no sense.

In the simulator there are 194 real geofence entries and exits during the day. A platform that treats everything as “now” raises 240 alerts: 140 on time, 54 about things that happened a while ago raised as if they were happening at that moment, and 46 false ones, from history and duplicates mixing with the present and making the device seem to go in and out. It also stores the 436 duplicates. Those false alerts depend on how the device resends: if everything goes out in order, they drop to 30; with no lost acknowledgments, to 16; with both, to zero.

A platform that separates history and present drops the duplicates, stores the history in its place and only alerts on the present. It raises 156 alerts: 140 on time and 16 late ones that say they are late. No false ones, though that is by construction: the simulator only carries the failures we put in it. And the route, rebuilt with each position’s time, recovers 186 of the 194 entries and exits correctly timed; the 8 missing ones had their records wiped by the device’s full memory, and no platform can fix that.

What I would do today

Treat disconnection as a normal condition, not an exception. The design starts by assuming the connection will fail.

Split the responsibilities into layers:

• Reception: accept, validate and identify the device’s messages.
• Messaging: decouple communication from processing with queues.
• Persistence: store idempotently, without duplicates.
• Temporal processing: tell historical from current.
• Business rules: evaluate geofences, alarms and operating conditions while respecting the time context.
• Presentation: show both the last known state and the history correctly.

Store the event time and the reception time separately. A historical position keeps its original time, if it is valid, and reception is recorded apart. That way the route can be rebuilt without altering the chronology.

Tell a current condition from a historical event detected later. History rebuilds the route; the present triggers alerts. If something happened with no signal and is discovered on reconnection, it is recorded as what it is.

Measure where it gets stuck. Separating reception, queue and processing makes it possible to see at which point in the flow a delay happens, instead of guessing.

In our current architectures we work with reception gateways, RabbitMQ, processing services, databases and event distribution over Socket.IO. That decouples reception from processing, although on its own it does not guarantee that a device kept everything during the disconnection.

To try it with another fleet, another network outage, another memory size or another processing rate, there is the store-and-forward simulator: it runs in the browser and compares both platforms with the three times of each data point.

Receiving information is not the same as receiving timely information, and processing a message correctly does not guarantee we are reading it in the right time context. A good telemetry platform does not just survive devices going offline: it recovers their history without confusing it with the present.

Israel Negrete Lepe is an electronics engineer. Twenty-five years making different systems work together under real conditions: from the sensor, the vehicle or the transaction to the backend, the data, security and operations. Today, increasingly, edge AI.