Verifying replica integrity
Multi-server installations are
accessed through the p4 journaldbchecksums
command,
and their behavior is controlled by three configurables:
, rpl.checksum.auto
, and
rpl.checksum.change
rpl.checksum.table
.
When you run p4 journaldbchecksums
against a
specific database table (or the set of tables associated with one of the
levels predefined by the rpl.checksum.auto
configurable),
the upstream server writes a journal note containing table checksum
information. Downstream replicas, upon receiving this journal note, verify these checksums and record their results in the
structured log for integrity-related events.
These checks are also performed whenever the journal is rotated. In addition, triggers allow you to take action when journals are rotated. See Triggering on journal rotation.
Administrators who have one or more replica servers deployed should
enable structured logging for integrity events, set the
rpl.checksum.*
configurables for their replica servers, and
regularly monitor the logs for integrity events.
Configuration
Structured server logging must be enabled on every server, with at least
one log recording events of type integrity
, for example:
$ p4 configure set serverlog.file.8=integrity.csv
After you have enabled structured server logging, set the following configurables to the desired levels of integrity checking:
Best practice for most sites is a balance between performance and log size:
p4 configure set rpl.checksum.auto=1 (or
2
for additional verification that is unlikely to vary
between an upstream server and its replicas.)
p4 configure set rpl.checksum.change=2 (this setting checks the integrity of every changelist, but only writes to the log if there is an error.)
p4 configure set rpl.checksum.table=1 (this setting instructs replicas to verify table integrity on scan or unload operations, but only writes to the log if there is an error.)
Valid settings for rpl.checksum.auto
are:
rpl.checksum.auto
|
Database tables checked with every journal rotation |
---|---|
|
No checksums are performed. |
|
Verify only the most important system and revision tables:
|
|
Verify all database tables from level 1, plus:
|
|
Verify all metadata, including metadata that is likely to differ, especially when comparing an upstream server with a build-farm or edge-server replica. |
Valid settings for rpl.checksum.change
are:
rpl.checksum.change
|
Verification performed with each changelist |
---|---|
|
Perform no verification. |
|
Write a journal note when a |
|
Replica verifies changelist summary, and writes to
|
|
Replica verifies changelist summary, and writes to integrity log even when the changelist does match. |
Valid settings for rpl.checksum.table
are:
rpl.checksum.table
|
Level of table verification performed |
---|---|
|
Table-level checksumming only. |
|
When a table is unloaded or scanned, journal notes are written.
These notes are processed by the replica and are logged to
|
|
When a table is unloaded or scanned, journal notes are written, and the results of journal note processing are logged even if the results match. |