Clocks in a Linux System

There are two types of date-time clocks:

The Hardware Clock: This clock is an independent hardware device, with its own power domain (battery, capacitor, etc), that operates when the machine is powered off, or even unplugged.

On an ISA compatible system, this clock is specified as part of the ISA standard. A control program can read or set this clock only to a whole second, but it can also detect the edges of the 1 second clock ticks, so the clock actually has virtually infinite precision.

This clock is commonly called the hardware clock, the real time clock, the RTC, the BIOS clock, and the CMOS clock. Hardware Clock, in its capitalized form, was coined for use by hwclock. The Linux kernel also refers to it as the persistent clock.

Some non-ISA systems have a few real time clocks with only one of them having its own power domain. A very low power external I2C or SPI clock chip might be used with a backup battery as the hardware clock to initialize a more functional integrated real-time clock which is used for most other purposes.

The System Clock: This clock is part of the Linux kernel and is driven by a timer interrupt. (On an ISA machine, the timer interrupt is part of the ISA standard.) It has meaning only while Linux is running on the machine. The System Time is the number of seconds since 00:00:00 January 1, 1970 UTC (or more succinctly, the number of seconds since 1969 UTC). The System Time is not an integer, though. It has virtually infinite precision.

The System Time is the time that matters. The Hardware Clock’s basic purpose is to keep time when Linux is not running so that the System Clock can be initialized from it at boot. Note that in DOS, for which ISA was designed, the Hardware Clock is the only real time clock.

It is important that the System Time not have any discontinuities such as would happen if you used the date(1) program to set it while the system is running. You can, however, do whatever you want to the Hardware Clock while the system is running, and the next time Linux starts up, it will do so with the adjusted time from the Hardware Clock. Note: currently this is not possible on most systems because hwclock --systohc is called at shutdown.

The Linux kernel’s timezone is set by hwclock. But don’t be misled — almost nobody cares what timezone the kernel thinks it is in. Instead, programs that care about the timezone (perhaps because they want to display a local time for you) almost always use a more traditional method of determining the timezone: They use the TZ environment variable or the /etc/localtime file, as explained in the man page for tzset(3). However, some programs and fringe parts of the Linux kernel such as filesystems use the kernel’s timezone value. An example is the vfat filesystem. If the kernel timezone value is wrong, the vfat filesystem will report and set the wrong timestamps on files. Another example is the kernel’s NTP '11 minute mode'. If the kernel’s timezone value and/or the persistent_clock_is_local variable are wrong, then the Hardware Clock will be set incorrectly by '11 minute mode'. See the discussion below, under Automatic Hardware Clock Synchronization by the Kernel.

hwclock sets the kernel’s timezone to the value indicated by TZ or /etc/localtime with the --hctosys or --systz functions.

The kernel’s timezone value actually consists of two parts: 1) a field tz_minuteswest indicating how many minutes local time (not adjusted for DST) lags behind UTC, and 2) a field tz_dsttime indicating the type of Daylight Savings Time (DST) convention that is in effect in the locality at the present time. This second field is not used under Linux and is always zero. See also settimeofday(2).

Hardware Clock Access Methods

hwclock uses many different ways to get and set Hardware Clock values. The most normal way is to do I/O to the rtc device special file, which is presumed to be driven by the rtc device driver. Also, Linux systems using the rtc framework with udev, are capable of supporting multiple Hardware Clocks. This may bring about the need to override the default rtc device by specifying one with the --rtc option.

However, this method is not always available as older systems do not have an rtc driver. On these systems, the method of accessing the Hardware Clock depends on the system hardware.

On an ISA compatible system, hwclock can directly access the "CMOS memory" registers that constitute the clock, by doing I/O to Ports 0x70 and 0x71. It does this with actual I/O instructions and consequently can only do it if running with superuser effective userid. This method may be used by specifying the --directisa option.

This is a really poor method of accessing the clock, for all the reasons that userspace programs are generally not supposed to do direct I/O and disable interrupts. hwclock provides it for testing, troubleshooting, and because it may be the only method available on ISA systems which do not have a working rtc device driver.

The Adjust Function

The Hardware Clock is usually not very accurate. However, much of its inaccuracy is completely predictable - it gains or loses the same amount of time every day. This is called systematic drift. hwclock's --adjust function lets you apply systematic drift corrections to the Hardware Clock.

It works like this: hwclock keeps a file, /etc/adjtime, that keeps some historical information. This is called the adjtime file.

Suppose you start with no adjtime file. You issue a hwclock --set command to set the Hardware Clock to the true current time. hwclock creates the adjtime file and records in it the current time as the last time the clock was calibrated. Five days later, the clock has gained 10 seconds, so you issue a hwclock --set --update-drift command to set it back 10 seconds. hwclock updates the adjtime file to show the current time as the last time the clock was calibrated, and records 2 seconds per day as the systematic drift rate. 24 hours go by, and then you issue a hwclock --adjust command. hwclock consults the adjtime file and sees that the clock gains 2 seconds per day when left alone and that it has been left alone for exactly one day. So it subtracts 2 seconds from the Hardware Clock. It then records the current time as the last time the clock was adjusted. Another 24 hours go by and you issue another hwclock --adjust. hwclock does the same thing: subtracts 2 seconds and updates the adjtime file with the current time as the last time the clock was adjusted.

When you use the --update-drift option with --set or --systohc, the systematic drift rate is (re)calculated by comparing the fully drift corrected current Hardware Clock time with the new set time, from that it derives the 24 hour drift rate based on the last calibrated timestamp from the adjtime file. This updated drift factor is then saved in /etc/adjtime.

A small amount of error creeps in when the Hardware Clock is set, so --adjust refrains from making any adjustment that is less than 1 second. Later on, when you request an adjustment again, the accumulated drift will be more than 1 second and --adjust will make the adjustment including any fractional amount.

hwclock --hctosys also uses the adjtime file data to compensate the value read from the Hardware Clock before using it to set the System Clock. It does not share the 1 second limitation of --adjust, and will correct sub-second drift values immediately. It does not change the Hardware Clock time nor the adjtime file. This may eliminate the need to use --adjust, unless something else on the system needs the Hardware Clock to be compensated.

The Adjtime File

While named for its historical purpose of controlling adjustments only, it actually contains other information used by hwclock from one invocation to the next.

The format of the adjtime file is, in ASCII:

Line 1: Three numbers, separated by blanks: 1) the systematic drift rate in seconds per day, floating point decimal; 2) the resulting number of seconds since 1969 UTC of most recent adjustment or calibration, decimal integer; 3) zero (for compatibility with clock(8)) as a floating point decimal.

Line 2: One number: the resulting number of seconds since 1969 UTC of most recent calibration. Zero if there has been no calibration yet or it is known that any previous calibration is moot (for example, because the Hardware Clock has been found, since that calibration, not to contain a valid time). This is a decimal integer.

Line 3: "UTC" or "LOCAL". Tells whether the Hardware Clock is set to Coordinated Universal Time or local time. You can always override this value with options on the hwclock command line.

You can use an adjtime file that was previously used with the clock(8) program with hwclock.

Automatic Hardware Clock Synchronization by the Kernel

You should be aware of another way that the Hardware Clock is kept synchronized in some systems. The Linux kernel has a mode wherein it copies the System Time to the Hardware Clock every 11 minutes. This mode is a compile time option, so not all kernels will have this capability. This is a good mode to use when you are using something sophisticated like NTP to keep your System Clock synchronized. (NTP is a way to keep your System Time synchronized either to a time server somewhere on the network or to a radio clock hooked up to your system. See RFC 1305.)

If the kernel is compiled with the '11 minute mode' option it will be active when the kernel’s clock discipline is in a synchronized state. When in this state, bit 6 (the bit that is set in the mask 0x0040) of the kernel’s time_status variable is unset. This value is output as the 'status' line of the adjtimex --print or ntptime commands.

It takes an outside influence, like the NTP daemon to put the kernel’s clock discipline into a synchronized state, and therefore turn on '11 minute mode'. It can be turned off by running anything that sets the System Clock the old fashioned way, including hwclock --hctosys. However, if the NTP daemon is still running, it will turn '11 minute mode' back on again the next time it synchronizes the System Clock.

If your system runs with '11 minute mode' on, it may need to use either --hctosys or --systz in a startup script, especially if the Hardware Clock is configured to use the local timescale. Unless the kernel is informed of what timescale the Hardware Clock is using, it may clobber it with the wrong one. The kernel uses UTC by default.

The first userspace command to set the System Clock informs the kernel what timescale the Hardware Clock is using. This happens via the persistent_clock_is_local kernel variable. If --hctosys or --systz is the first, it will set this variable according to the adjtime file or the appropriate command-line argument. Note that when using this capability and the Hardware Clock timescale configuration is changed, then a reboot is required to notify the kernel.

hwclock --adjust should not be used with NTP '11 minute mode'.

ISA Hardware Clock Century value

There is some sort of standard that defines CMOS memory Byte 50 on an ISA machine as an indicator of what century it is. hwclock does not use or set that byte because there are some machines that don’t define the byte that way, and it really isn’t necessary anyway, since the year-of-century does a good job of implying which century it is.

If you have a bona fide use for a CMOS century byte, contact the hwclock maintainer; an option may be appropriate.

Note that this section is only relevant when you are using the "direct ISA" method of accessing the Hardware Clock. ACPI provides a standard way to access century values, when they are supported by the hardware.

Keeping Time without External Synchronization

This discussion is based on the following conditions:

Whether maintaining precision time with NTP daemon or not, it makes sense to configure the system to keep reasonably good date-time on its own.

The first step in making that happen is having a clear understanding of the big picture. There are two completely separate hardware devices running at their own speed and drifting away from the 'correct' time at their own rates. The methods and software for drift correction are different for each of them. However, most systems are configured to exchange values between these two clocks at startup and shutdown. Now the individual device’s time keeping errors are transferred back and forth between each other. Attempt to configure drift correction for only one of them, and the other’s drift will be overlaid upon it.

This problem can be avoided when configuring drift correction for the System Clock by simply not shutting down the machine. This, plus the fact that all of hwclock's precision (including calculating drift factors) depends upon the System Clock’s rate being correct, means that configuration of the System Clock should be done first.

The System Clock drift is corrected with the adjtimex(8) command’s --tick and --frequency options. These two work together: tick is the coarse adjustment and frequency is the fine adjustment. (For systems that do not have an adjtimex package, ntptime -f ppm may be used instead.)

Some Linux distributions attempt to automatically calculate the System Clock drift with adjtimex's compare operation. Trying to correct one drifting clock by using another drifting clock as a reference is akin to a dog trying to catch its own tail. Success may happen eventually, but great effort and frustration will likely precede it. This automation may yield an improvement over no configuration, but expecting optimum results would be in error. A better choice for manual configuration would be adjtimex's --log options.

It may be more effective to simply track the System Clock drift with sntp, or date -Ins and a precision timepiece, and then calculate the correction manually.

After setting the tick and frequency values, continue to test and refine the adjustments until the System Clock keeps good time. See adjtimex(2) for more information and the example demonstrating manual drift calculations.

Once the System Clock is ticking smoothly, move on to the Hardware Clock.

As a rule, cold drift will work best for most use cases. This should be true even for 24/7 machines whose normal downtime consists of a reboot. In that case the drift factor value makes little difference. But on the rare occasion that the machine is shut down for an extended period, then cold drift should yield better results.

Steps to calculate cold drift:

1

2

3

4

5

6

Note: if step 6 uses --systohc, then the System Clock must be set correctly (step 6a) just before doing so.

Having hwclock calculate the drift factor is a good starting point, but for optimal results it will likely need to be adjusted by directly editing the /etc/adjtime file. Continue to test and refine the drift factor until the Hardware Clock is corrected properly at startup. To check this, first make sure that the System Time is correct before shutdown and then use sntp, or date -Ins and a precision timepiece, immediately after startup.

LOCAL vs UTC

Keeping the Hardware Clock in a local timescale causes inconsistent daylight saving time results:

The Hardware Clock on an ISA compatible system keeps only a date and time, it has no concept of timezone nor daylight saving. Therefore, when hwclock is told that it is in local time, it assumes it is in the 'correct' local time and makes no adjustments to the time read from it.

Linux handles daylight saving time changes transparently only when the Hardware Clock is kept in the UTC timescale. Doing so is made easy for system administrators as hwclock uses local time for its output and as the argument to the --date option.

POSIX systems, like Linux, are designed to have the System Clock operate in the UTC timescale. The Hardware Clock’s purpose is to initialize the System Clock, so also keeping it in UTC makes sense.

Linux does, however, attempt to accommodate the Hardware Clock being in the local timescale. This is primarily for dual-booting with older versions of MS Windows. From Windows 7 on, the RealTimeIsUniversal registry key is supposed to be working properly so that its Hardware Clock can be kept in UTC.

POSIX vs 'RIGHT'

A discussion on date-time configuration would be incomplete without addressing timezones, this is mostly well covered by tzset(3). One area that seems to have no documentation is the 'right' directory of the Time Zone Database, sometimes called tz or zoneinfo.

There are two separate databases in the zoneinfo system, posix and 'right'. 'Right' (now named zoneinfo-leaps) includes leap seconds and posix does not. To use the 'right' database the System Clock must be set to (UTC + leap seconds), which is equivalent to (TAI - 10). This allows calculating the exact number of seconds between two dates that cross a leap second epoch. The System Clock is then converted to the correct civil time, including UTC, by using the 'right' timezone files which subtract the leap seconds. Note: this configuration is considered experimental and is known to have issues.

To configure a system to use a particular database all of the files located in its directory must be copied to the root of /usr/share/zoneinfo. Files are never used directly from the posix or 'right' subdirectories, e.g., TZ='right/Europe/Dublin'. This habit was becoming so common that the upstream zoneinfo project restructured the system’s file tree by moving the posix and 'right' subdirectories out of the zoneinfo directory and into sibling directories:

/usr/share/zoneinfo, /usr/share/zoneinfo-posix, /usr/share/zoneinfo-leaps

Unfortunately, some Linux distributions are changing it back to the old tree structure in their packages. So the problem of system administrators reaching into the 'right' subdirectory persists. This causes the system timezone to be configured to include leap seconds while the zoneinfo database is still configured to exclude them. Then when an application such as a World Clock needs the South_Pole timezone file; or an email MTA, or hwclock needs the UTC timezone file; they fetch it from the root of /usr/share/zoneinfo , because that is what they are supposed to do. Those files exclude leap seconds, but the System Clock now includes them, causing an incorrect time conversion.

Attempting to mix and match files from these separate databases will not work, because they each require the System Clock to use a different timescale. The zoneinfo database must be configured to use either posix or 'right', as described above, or by assigning a database path to the TZDIR environment variable.