(one of Keith's Useful Information pages.)
What we needed
As part of our work on the NetSem project to formalise TCP, we accumulate trace data consisting of system calls and returns and TCP segments emitted and received. We then compare this data against the predictions of our model, to determine if our model is correct.
This exercise requires precise timing data, both because we are
interested in the length of time taken by system calls and the delays
between segments, and because the program that performs the system
calls is different from (and may be on a different machine from) the
one that observes segments on the wire. Correctly merging the two
traces requires high accuracy in timing - of the order of
100 microsecond.
Accurate timing on modern operating systems
This is pretty easy on most modern operating systems - Linux and
BSD both have implementations of NTP, and when they are running, the gettimeofday(2)
call gives the current time to microsecond accuracy (with a known
offset from UTC, obtained by invoking ntpq
).
Timing on Windows
Windows, however, is another story. Windows XP certainly has available a good implementation of NTP (how to install it), which will get the system time set accurately as we require (note that Windows XP has a built-in SNTP client, which can be set to run periodically in order to provide roughly-accurate time sufficient for typical home use). And the GetSystemTimeAsFileTime() call returns the current time in hectonanoseconds (100-nanosecond units) since 1601-01-01T00:00:00Z (11644473600 seconds before the start of the Unix epoch on 1970-01-01T00:00:00Z). Isn't this enough?
Well, no. While the return value is precise to 100ns, it is only updated at each timer tick - approximately 64 times per second on WinXP, or once every 15.625 milliseconds*. While NTP ensures that when it is updated it is updated to the correct time with very high accuracy, between those ticks you have no information. Unlike other modern operating systems, Windows XP makes no attempt to interpolate the return value between timer ticks.
*Note: The precise interval is configurable, via either
the Windows multimedia support library (timeBeginPeriod
etc.) or the undocumented system calls NTSetTimerResolution
and friends. This still doesn't provide the high-precision timing
we're after, though.
High-frequency counters
There are two ways of obtaining some higher-frequency timing data:
QueryPerformanceCounter()
and the Intel IA32 instruction RDTSC.
In fact, in Windows XP the former is a wrapper around the latter. RDTSC reads the
value of a 64-bit counter (the Time Stamp Counter) on the CPU that is
incremented every clock cycle. This is quite fast enough to give us
the precision we need. Windows' QueryPerformanceCounter() simply
ensures we get a stable value even on a multiprocessor system (and
possibly corrects for SpeedStep frequency changing technology; I'm not
sure). The companion call QueryPerformanceFrequency() obtains the
nominal frequency of this counter.
Calibrating the counters
But there are two problems - (i) the true frequency isn't exactly the nominal frequency, and (ii) there is no connection between the timestamp/performance count and the Windows timer-tick clock. Thus we know neither the scale nor the origin. Ugh!
The scale (frequency) is relatively easy to calibrate: obviously, read the counter value, wait a long time (in timer ticks, for which we know the precise duration from NTP), read the counter value again, and divide.
Calibrating the origin is rather harder. If you do a Sleep(),
you might hope that when you regain control you are probably pretty
soon after a clock tick. So assume that, reading the counter
immediately after a Sleep(), and repeat several times, taking the
earliest counter value recorded to be the counter value at the tick.
This method will have a small but hopefully constant offset; it's the
best we can do, at least in user space (more may be possible as a
driver, using the DDK KeQuery*
functions; we haven't investigated this).
Accurate timing on Windows, at last
Once both values are known - the real frequency of the counter in
counts per second, and the precise counter value at a known moment in
time - the precise time can be obtained at any moment by reading the
counter and performing the computation. To correct for errors in the
origin calculation (and avoid overflow problems when scaling time
intervals measured in counts), it is wise to recalibrate at regular
intervals.
Source code
Actually getting all this to work took a while, so to save reinvention of the wheel, we present our code here. This code is subject to a BSD-style open source license, and is provided without any warranty of fitness for any purpose whatsoever. It did work for us, though - we hope you will find it useful too!
The code has only been tested using Visual C++ 7 on Windows XP, although it should work on any Win32 platform.
There are two parts: TSCcal is a standalone calibration utility, and TSCtime is a module providing accurate timing calls to your application. Both use RDTSC rather than QueryPerformanceCounter(), because we found the latter took around 300 clock cycles to execute, whereas RDTSC is just a single instruction. This probably isn't important, and it should be obvious how to change it if find you are getting anomalous results on your system.
TSCcal.exe is a program that calibrates the
timestamp counter and stores the determined frequency in the registry
(at HKEY_LOCAL_MACHINE\SOFTWARE\NetSem\TTHEE
, in
tscfreq
), along with (for information) the standard
deviation of the measurements taken and the date and time of the
measurement. It takes two arguments on the command line: the number
of measurements to take, and the approximate length of each
measurement in milliseconds. I usually use 10 and 100000
respectively: ten measurements of one hundred seconds. This gives a
standard deviation of less than one part in a million.
TSCtime.obj/TSCtime.h is a module that can be linked
into your application, providing
gethectonanotime_first()
,
gethectonanotime_last()
, and
gethectonanotime_norecal()
, along with a few other useful
bits and pieces. The functions return an accurate time (in
hectonanoseconds since the Windows epoch, just like
GetSystemTimeAsFileTime()); they differ in when recalibration is done.
gethectonanotime_first()
returns the time when it was
called, but may delay before returning.
gethectonanotime_last()
returns the time when it returns,
but may delay before that. In both cases, the delay (of up to around
100ms) is recalibration being performed, and this is done every 100
seconds. gethectonanotime_norecal()
always returns
quickly, and never recalibrates. They rely on the registry values
written by TSCcal.exe, and also on an additional value under the same
key, ugly_hack_offset
, which is a string value parsed as
a signed decimal int64 offset (in hectonanoseconds) to be applied to
all times returned by TSCtime; this defaults to zero, but may be
useful in some circumstances.
Appendix: How to install NTP on Windows
It's not immediately obvious how to install the Windows implementation of NTP. Here is a step-by-step guide for Windows XP.
C:\Program Files\NTP
).
Instsrv "C:\Program Files\NTP\ntpd.exe"This installs the NTP dæmon as a service.
$ALT_CONFIG_FILE %windir%\ntp.conf $CONFIG_FILE %windir%\system32\drivers\etc\ntp.conf(note that while the
-c
parameter to ntpd certainly
works, there is no way to make Windows remember service startup
parameters for future starts; they apply only to this press of the
Start button).
So start up Notepad and write the configuration file, saving it in
C:\WINDOWS\system32\drivers\etc\ntp.conf
(or wherever
your Windows directory is). A sample configuration file might be:
# Configuration file for NTP (change the server appropriately!) server ntp.mydomain.com driftfile c:/WINDOWS/system32/drivers/etc/ntp.drift logfile c:/WINDOWS/system32/drivers/etc/ntp.log logconfig =all
ntpq
followed by rl
will tell you the
current NTP status, if the service is running.
We hope you find this code useful. If you encounter any problems (or if you use it successfully without any!) please contact us and let us know.
--Keith Wansbrough and the NetSem team, 2003-09-25.
[Useful Information] / [Home page]
--KW 8-)