How accurately can latency be measured?

The accuracy of a latency measurement derives directly from that of the timestamps used to construct it, which in turn is determined principally by two factors:

a) the fidelity with which the timestamps reflect the event of interest, and
b) the quality of the clock from which the timestamps are read.

The two primary sources of timestamps that are used to measure latency in HFT are software instrumentation and network capture. Other more exotic timestamping mechanisms have been used, but none has proven as effective or accurate.

  1. Software timestamps offer great flexibility, as developers can add code to generate timestamps close to any events of interest that occur in the software. They do however suffer a number of drawbacks:

    • Timestamping code adds latency - this can be made very small, adding as little as a few dozen nanoseconds to the critical path if done optimally, but is still non-zero.
    • Timestamping code cannot run synchronously with the event of interest - it must be run in the same thread as the event being timestamped, and hence is separated in time from the that event. Techniques such as disabling CPU interrupts can be used to minimise the possibility of the event and the code timestamping it from being excessively separated, but these are not always practical.
    • Out-of-order execution can make it uncertain exactly when timestamping code is run, and mitigations can increase the latency of that code. For example, the RDTSC instruction is very cheap, but RDTSCP can be two to three times slower.
    • The biggest challenge for software timestamps is often the quality of the clock from which timestamps are read: the most common time synchronisation mechanisms available to HFT systems are NTP or PTP. PTP usually provides a better time-signal than NTP, but it can be challenging to achieve sub-microsecond synchronization consistently.
  2. Network timestamps can be generated any time messages are exchanged over the network. Although this allows fewer opportunities for acquiring timestamps than software instrumentation, the fact that all counterparty transactions and exchange of data in electronic trading take place over the network ensures that all important events can be timestamped on the network.
    Furthermore network timestamping addresses the problems with software timestamps noted above:
    • Network timestamping is done in dedicated hardware and hence adds no latency.
    • The copying of packets is done in hardware, and timestamping can be done either as part of the replication or a deterministic length of time afterwards, so there is no ambiguity about when network events occur.
    • While PTP is also widely used for synchronization of network timestamping equipment, higher-precision and more reliable options (such as 1PPS over coaxial cable) are also available.

In summary, with appropriate equipment and engineering, it is feasible in HFT to measure latency with an accuracy of under 100ns.