Helix Core is now P4. Product user interfaces will be updated soon. Learn more >>

P4 Server Administration Documentation | 2025.2

# Filter during replication or edge-to-edge chaining

For performance reasons, you might want to ensure that replication occurs where necessary. Rules for replica filtering are therefore useful.

# Filtering rules

As part of an HA/DR solution, you typically want to ensure that all the metadata Information that P4 Server maintains, such as who created file revisions in the depot, whether the file is a 'lazy copy,, the current state of client workspaces, protections, groups, users, labels, streams, and branches. Metadata is stored in the server database and is separate from the 'archive files' that users submit from their client workspace into the depot. and all the versioned files Source files stored in the depot, including one or more revisions to each file. Also known as archive files, archives, and depot files. Versioned files typically use the naming convention 'filename,v' or '1.changelist.gz'. are replicated. In most other use cases, particularly build servers and forwarding replicas, this leads to a great deal of redundant data being transferred.

It is often advantageous to configure your replica servers to filter data on client workspaces and file revisions.
For example:

- Developers working on one project at a remote site do not typically need to know the state of every client workspace at other sites where other projects are being developed.
- Build servers don’t require access to the endless stream of changes to office documents and spreadsheets associated with a typical large enterprise.
- In the case of edge-to-edge chaining, the outer edge server A server that is part of a commit-edge environment that can independently support work in progress for locally-bound clients, thereby reducing the load on the commit server. might need only a subset of what the inner edge server has.

Filters only apply for metadata and files that are handled by the p4 pull threads defined in the startup.N configurables.

Changes to the filtering rules in the spec of a replica server or edge server affect the replication of server data from the target server The immediately upstream server for replica servers, edge servers, standby servers, proxies and brokers. See also 'upstream server' and 'central server'. to the replica A P4 Server that automatically maintains a full or partial copy of the central server's metadata and that might contain related file content. The replica copies by using 'p4 pull' or 'p4 journalcopy'. A replica can be used as a backup server for disaster recovery. server or edge server.

In the case of edge servers, the filters affect what the upstream server Any server in the inward direction, that is, toward the central server. For example, in an edge-to-edge configuration with a commit, edge1, and edge2, both edge1 and the commit server are upstream servers for edge2. See also 'central server'. replicates to the downstream replicas, where the upstream server is either the commit server The innermost P4 Server server in a topology with one or more edge servers. or an upstream edge server in a chain of edge servers.

Changing a filter to:

- exclude file revision metadata does not cause versioned files to be removed from the edge or replica server.

- include file revision metadata uses additional disk storage when the relevant versioned files are transferred to the edge or replica server.

You can configure filtering rules with three fields of the `p4 server` form: `ClientDataFilter`, `RevisionDataFilter`, and `ArchiveDataFilter`.

## When filtering rules are applied

When filtering rules are applied depends on the version of the server.

2025.2 and later | Prior to 2025.2
--- | ---
`RevisionDataFilter` and `ClientDataFilter` fields:
Changes that expand filtering rules to include previously excluded file revision and client metadata cause the replica (or edge) to be updated with retrospective metadata for newly included paths. Changes that restrict filtering rules to exclude previously included file revision and client metadata cause the replica (or edge) to purge file revision and client metadata for the newly excluded paths. The retrospective metadata updates and purging occur the next time that either of the following runs: journal pull thread `p4 admin replica-filter-reconcile` | Changes that expand filtering rules to include previously excluded file revision and client metadata take effect only on future updates to file revision and client metadata. Changes that restrict filtering rules to exclude previously included file revision and client metadata do not purge file revision and client metadata on the replica for the newly excluded content.
`ArchiveDataFilter` field:
Changes that expand filtering rules to include previously excluded versioned files paths take effect only on future updates. Changes that restrict filtering rules to exclude previously included versioned files paths do not purge versioned file data on the replica for newly excluded content. | Changes that expand filtering rules to include previously excluded versioned files paths take effect only on future updates. Changes that restrict filtering rules to exclude previously included versioned files paths do not purge versioned file data on the replica for newly excluded content.

## Replica reconciliation versus reseeding

For ease of administration, P4 Server 2025.2 introduced the feature of reconciliation without reseeding the replica.

2025.2 and later | Prior to 2025.2
--- | ---
Reconciliation without reseeding: | Reseeding:
P4 Server does not require recreating (reseeding) the replica if the replica filter changes. This is because of the feature of automatic replica filter reconciliation. | P4 Server requires recreating (reseeding) the replica if the replica filter changes. This is because changes to the filtering rules in the server specification are not applied retrospectively.
The replica filter reconcile feature combines the ability the purge unnecessary records and fetch records that have become necessary. In other words, changes to `RevisionDataFilter` or `ClientDataFilter` rules cause the replica (or edge) to be updated to: include any file revision metadata from newly added paths, and remove any file revision metadata from paths now excluded. Changes in replication might have a performance impact. By default, the journal pull thread automatically performs the reconciliation when it detects a change in the replica filter. These configurables control the behavior of the reconciliation:
Purge database records | Fetch database records
--- | ---
To change the number of threads used when purging multiple database tables in parallel, use the rpl.filter.restrict.threads configurable. | To change the number of threads used to process database tables in parallel, use the rpl.filter.expand.threads configurable.
To change the number of database records that are processed in parallel, use the rpl.filter.restrict.batch configurable. | To change the block size used for calculating the checksum to compare to database records to those of the upstream server, use the rpl.filter.expand.batch configurable. Blocks are used to avoid dumping the entire table to the journal if only a few records are missing on the replica.
To disable automatic purging, set the rpl.filter.restrict configurable to `0`. | To disable automatic fetching, set the rpl.filter.expand configurable to `0`.
An administrator can run a manual replica filter reconciliation from the command line, which provides an option to specify one or more specific tables the command applies to. To learn more, see p4 admin replica-filter-reconcile in the P4 CLI Reference. | Sensitive or unneeded versioned files can be removed by running `p4 cachepurge` before reseeding the replica or edge server. Reseed the replica First, on the target server, create a filtered checkpoint:
    p4d -r /p4/london -P site1-1668 -jd myCheckpoint
where `/p4/london` is the P4ROOT directory of the target server, and `site1-1668` is the name of the server spec containing filter rules used to create a filtered checkpoint. Then, replay that checkpoint file on the replica server:
    p4d -r /p4/replica -jr myCheckpoint
Referring to the example server spec in Replica scenario, the effects of filtering are: the exclusion of client metadata for workspace names matching `site2-ws-*` and `site3-ws-*` that pull threads no longer transfer versioned files with the `.mp4` suffix.

The `p4 verify -t` command does not respect the ArchiveDataFilter that applies to a specific server instance. If metadata exists on that instance and `p4 verify -t` detects the archive file is missing within the `file[revRange]` provided, the `p4 verify -t` command will cause the file to be scheduled for transfer. This can have a performance impact along with populating the server with archive files it was not originally intended to contain.

## Two ways to filter

Exclude database tables | Filter by fields
--- | ---
The simplest way to filter metadata is by using the `-T tableexcludelist` option with the `p4 pull` command. For example, if you know that a build server has no need to refer to any of your users' have lists or the state of their client workspaces, you can filter out `db.have` and `db.working` entirely with `p4 pull -T db.have,db.working`. Excluding entire database tables is a coarse-grained method of managing the amount of data passed between servers, requires some knowledge of which tables are most likely to be referred to during P4 Server command operations, and offers no means of control over which versioned files are replicated. | You can have fine-grained control over what data is replicated by using the `ClientDataFilter:`, `RevisionDataFilter:`, and `ArchiveDataFilter` fields of the `p4 server` form. These fields enable you to replicate only a subset of the server metadata and versioned files to a replica or edge. For this feature to work, the value of the Services: field in the server spec must be a value other than commit-server or standard. Changes to `RevisionDataFilter` or `ClientDataFilter` rules cause the replica (or edge) to be updated to include any file revision metadata from newly added paths and remove any file revision metadata from paths now excluded. Changes in replication might have a performance impact.

### Replica scenario

Example: Filtering out client workspace data and files

If workspaces for users in each of three sites are named with `site[123]-ws-username`, a replica intended to act as partial backup for users at `site1` could be configured as follows:

Copy

    ServerID:       site1-1668
    Name:           site1-1668
    Type:           server
    Services:       replica
    Address:        tcp:site1bak:1668
    Description:
            Replicate all client workspace data, except the states of
            workspaces of users at sites 2 and 3.
            Automatically replicate .c files in anticipation of user
            requests. Do not replicate .mp4 video files, which tend
            to be large and impose high bandwidth costs.
    ClientDataFilter:
            //...
            -//site2-ws-*/...
            -//site3-ws-*/...
    RevisionDataFilter:
    ArchiveDataFilter:
            //....c
            -//....mp4

When you start the replica, your `p4 pull` metadata thread might resemble the following:

    p4 configure set "site1-1668#startup.1=pull -i 30"

In this configuration, only those portions of `db.have` that are associated with `site1` are replicated. All metadata concerning workspaces associated with `site2` and `site3` is ignored.

All file-related metadata is replicated. All files in the depot are replicated, except for those with the `.mp4` extension. Files ending in `.c` are transferred automatically to the replica when submitted.

### Build server scenario

Consider a build server scenario. The ongoing work of the organization (such as code, business documents, or videos) can be stored anywhere in the depot. In contrast, this build farm is dedicated to building releasable products, and therefore only needs a subset of the organization’s output:

Example: Replicating metadata and file contents for a subset of a depot

Releasable code is placed into `//depot/releases/...` and automated builds are based on these changes.

Copy

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    ServerID:       builder-1669
    Name:           builder-1669
    Type:           server
    Services:       build-server
    Address:        tcp:built:1669
    Description:
            Exclude all client workspace data
            Replicate only revisions in release branches
    RevisionDataFilter:
            //depot/releases/...
    ArchiveDataFilter:
            //depot/releases/...

### Exclude a subset of paths

If you want to exclude a subset of paths, ensure that the the inclusionary lines precede the exclusionary lines. For example,

    RevisionDataFilter:
        //...
        -//depot/releases/...

Your search for returned results