Verifying replica integrity

Multi-server installations are accessed through the p4 journaldbchecksums command, and their behavior is controlled by three configurables: rpl.checksum.auto, rpl.checksum.change, and rpl.checksum.table.

When you run p4 journaldbchecksums against a specific database table (or the set of tables associated with one of the levels predefined by the rpl.checksum.auto configurable), the upstream server writes a journal note containing table checksum information. Downstream replicas, upon receiving this journal note, verify these checksums and record their results in the structured log for integrity-related events.

These checks are also performed whenever the journal is rotated. In addition, triggers allow you to take action when journals are rotated. See Triggering on journal rotation.

Administrators who have one or more replica servers deployed should enable structured logging for integrity events, set the rpl.checksum.* configurables for their replica servers, and regularly monitor the logs for integrity events.

Configuration

Structured server logging must be enabled on every server, with at least one log recording events of type integrity, for example:

$ p4 configure set serverlog.file.8=integrity.csv

After you have enabled structured server logging, set the following configurables to the desired levels of integrity checking:

Best practice for most sites is a balance between performance and log size:

p4 configure set rpl.checksum.auto=1 (or 2 for additional verification that is unlikely to vary between an upstream server and its replicas.)

p4 configure set rpl.checksum.change=2 (this setting checks the integrity of every changelist, but only writes to the log if there is an error.)

p4 configure set rpl.checksum.table=1 (this setting instructs replicas to verify table integrity on scan or unload operations, but only writes to the log if there is an error.)

Valid settings for rpl.checksum.auto are:

rpl.checksum.auto Database tables checked with every journal rotation

0

No checksums are performed.

1

Verify only the most important system and revision tables:

db.archmap, db.config, db.depot, db.graphindex, db.graphperm, db.group, db.groupx, db.integed, db.integtx, db.ldap, db.object, db.protect, db.pubkey, db.ref, db.rev, db.revcx, db.revdx, db.revhx, db.revtx, db.stream, db.submodule, db.ticket, db.trigger, db.user

2

Verify all database tables from level 1, plus:

db.bodtext, db.bodtextcx, db.bodtexthx, db.counters, db.excl, db.fix, db.fixrev, db.haveview, db.ixtext, db.ixtexthx, db.job, db.logger, db.message, db.nameval, db.property, db.remote, db.repo, db.revbx, db.review, db.revsx, db.revux, db.rmtview, db.server, db.svrview, db.traits, db.uxtext

3

Verify all metadata, including metadata that is likely to differ, especially when comparing an upstream server with a build-farm or edge-server replica.

Valid settings for rpl.checksum.change are:

rpl.checksum.change Verification performed with each changelist

0

Perform no verification.

1

Write a journal note when a p4 submit, p4 fetch, p4 populate, p4 push, or p4 unzip command completes. The value of the rpl.checksum.change configurable will determine the level of verification performed for the command.

2

Replica verifies changelist summary, and writes to integrity.csv if the changelist does not match.

3

Replica verifies changelist summary, and writes to integrity log even when the changelist does match.

Valid settings for rpl.checksum.table are:

rpl.checksum.table Level of table verification performed

0

Table-level checksumming only.

1

When a table is unloaded or scanned, journal notes are written. These notes are processed by the replica and are logged to integrity.csv if the check fails.

2

When a table is unloaded or scanned, journal notes are written, and the results of journal note processing are logged even if the results match.