This documentation covers perfmerge version 2023.2
Perfmerge is a tool for merging two Perforce repositories to produce a third repository. Unlike its predecessor, perfmerge does not work with checkpoints, rather it accesses the database files directly and creates new database files in the target directory. These new database files contain the merged content.
Perfmerge is provided in the hope that it will be useful, and all use is subject to the following Terms and Conditions:
THIS SOFTWARE IS PROVIDED BY PERFORCE SOFTWARE, INC. 'AS IS' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL PERFORCE SOFTWARE, INC. BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Perfmerge is a support utility, not a Perforce product. If you encounter any problems using this software, please contact support-helix-core@perforce.com.
Merging Perforce databases is a messy business: there are any number of incompatibilities that may exist between the databases being merged; perfmerge tests for the existence of some of these issues, but it is unlikely that perfmerge can cover all the issues. The most important pre-condition is the version of perfmerge:
You must ensure that the platform and release of perfmerge are identical on both databases to be merged. Using a newer perfmerge on your database may irreversibly upgrade your database tables necessitating a recovery from checkpoint.
Perfmerge will currently refuse to merge your databases if:
Perfmerge reads the complete contents of both databases (sets of db.* files). It is thus similar to performing a checkpoint. The time taken to perform a perfmerge is thus likely to be more or less the sum of the time taken to checkpoint each of the two source databases (in practice it may be a little less - but try and find out on your hardware!) It is therefore recommended to perform this on the fastest hardware you have available (both CPU and disks - so SSD drives will help!)
If you haven't restored your db.* files from a checkpoint recently, then performance may be improved by doing so before the merge. It is also recommended to do some house cleaning and remove old or unused client workspaces to reduce the size of db.have in the source databases.
Although perfmerge doesn't usually need archive files to do its job, (You will copy them in a post-merge step ) There is an exception if you have a //.swarm depot and are merging servers containing Swarm data. Before you merge you should verify that the swarm attachment archive files are accessable on each server by running "p4 verify -q //.swarm/attachments/..." .
Be careful when you copy the archive files to the merged server. Do not overwrite the .swarm depot on the merged server with the .swarm depot from one of the source servers. Doing so will cause the merged server to lose all of the swarm attachment archive file renames which were made during the merge.
The facts that (a) jobs can be arbitrarily named, and (b) jobs can be based on a customized jobspec make the task of merging job information problematic. Perfmerge is able to merge job data from databases that use exactly the same jobspec, and that follow the default Perforce naming convention for their jobs (for example: job000001).
Job data from databases with differing jobspecs cannot be merged; however, you may choose to include only job information from one of the source databases in your merge result by using either of the '-jl' or '-jr' flags.
Jobs that are named using a local naming convention can be merged (assuming the jobspecs are identical), but they will not be renumbered: thus you should ensure that there are no job naming conflicts before merging.
With 2010.2 and later Perforce Servers, server configuration data set with the p4 configure command is stored within the Perforce database files in the db.config table. Note, server configuration data does not include counter data set using the p4 counter command. When merging databases that contain server configuration data, you must choose which database to take the server configuration data and history from by supplying the -cl or -cr option on the perfmerge command line. For example, to take the left database configuration data:
$ perfmerge -t merge -cl /perforce/serverA /perforce/serverB /perforce/serverCServer configuration data and history from the database not chosen will be written to the output database with its server name (P4NAME) value prepended with "left-" or "right-" accordingly. When merging databases that contain no server configuration data, the -cl or -cr option is not required. For example, if the configurations for the serverA and serverB databases in the example above were as follows:
Left database (/perforce/serverA):
$ p4 configure show allservers
any: monitor = 2
any: P4LOG = master.log
any: security = 1
any: server = 3
replica1: db.replication = readonly
replica1: lbr.replication = readonly
replica1: P4LOG = replica.log
replica1: P4TARGET = perforce:1666
replica1: serviceUser = service
replica1: startup.1 = pull -i 4
replica1: startup.2 = pull -u -i 4
replica1: startup.3 = pull -u -i 4
Right database (/perforce/serverB):
$ p4 configure show allservers
any: monitor = 1
any: security = 3
p4_1: net.mimcheck = 1
p4_1: P4LOG = log
p4_1: server = 1
The merged server configuration would look as follows if the -cl option was provided on the perfmerge command line to take the left database server configuration data:
Merged database (/perforce/serverC):
$ p4 configure show allservers
any: monitor = 2
any: P4LOG = master.log
any: security = 1
any: server = 3
replica1: db.replication = readonly
replica1: lbr.replication = readonly
replica1: P4LOG = replica.log
replica1: P4TARGET = perforce:1666
replica1: serviceUser = service
replica1: startup.1 = pull -i 4
replica1: startup.2 = pull -u -i 4
replica1: startup.3 = pull -u -i 4
right-any: monitor = 1
right-any: security = 3
right-p4_1: net.mimcheck = 1
right-p4_1: P4LOG = log
right-p4_1: server = 1
See 'p4 help configure' for further details on managing server configuration data.
If the servers being merged contain conflicting user data, perfmerge will produce the following error:
perfmerge error: The servers being merged contain conflicting user names! Either resolve the conflicts or specify which database to take conflicting user data from by supplying the -ul or -ur option on the perfmerge command line.To correct the problem, either resolve the user conflicts manually by removing users from one of the input databases, or add the -ul or -ur option to the perfmerge command line to specify which input database to use when user conflicts are encountered. For example, if you have conflicting user data and want to take user data from the left input database when user conflicts exist, add the -ul option to the perfmerge command line:
$ perfmerge -t merge -ul /perforce/serverA /perforce/serverB /perforce/serverCThis instructs perfmerge to take user data from the left database (serverA in this case) when user conflicts exist. All non-conflicting user data will be merged into the output database. Any public keys stored by the server (p4 pubkeys) for a particular user are also affected by the -ul and -ur options; if you specify -ul and a user conflict exists, only public keys for that user in the left input database will be copied to the output database.
With 2011.1 and later Perforce Servers, streams data may be present. When streams data is present, only offset merge types are supported. For example, attempting to run an append type merge against databases that contain streams data:
$ perfmerge -t append /perforce/serverA /perforce/serverB /perforce/serverCproduces the following error:
perfmerge error: The servers being merged contain streams data. Only offset type merges are supported when streams data is present.
With 2012.2 and later Perforce Servers, unload depots may be present. When unload depots are present, the naming of the unload depots and the presence of unloaded client or task stream data within unload depots may affect the merge process. The following rules apply:
p4 files -U //unload/client/... p4 files -U //unload/label/... p4 files -U //unload/stream/...For details on deleting or reloading clients, labels, or task streams, see the command reference for the p4 reload, p4 client, p4 label, and p4 stream commands.
perfmerge supports the merge of Swarm data using these options:
Merging right hand Swarm data requires an "offset" type merge
Merging swarm attachments requires access to the archive files, if the server has a //.swarm depot, please verify that "p4 verify //.swarm/attachments/..." runs cleanly on it before starting perfmerge
The presence of Commons, or Git Fusion data may affect the merge process. When Commons or Git Fusion data exists, merging is only allowed in the following cases:
Merging servers that are part of a Commit/Edge Server architecture introduces restrictions. The following rules should be followed:
When only one of the input servers is a Commit Server, you can perform the merge without decommissioning Edge Servers following the steps below. An offset merge type is required and the Commit Server can't have its changelists renumbered so it must be specified as the left input database to the Perfmerge process. Note, before following the steps below take full checkpoints of the Edge and Commit Servers prior to carrying out the Perfmerge process. In the event the merge doesn't go as planned, this leaves you with a recovery point to fall back to.
1. Prevent new server activity on the Commit Server and wait for the Edge Server to catch up to the Commit Server in terms of metadata and archive replication 2. Shut down the Commit and Edge Server 3. Checkpoint the Edge Server$ cd $P4ROOT $ p4d -r . -k "db.have,db.working,db.resolve,db.locks,db.revsh,db.workingx,db.resolvex,db.view,db.label,db.revsx,db.revux" -jd edge.ckp4. Use Perfmerge to merge the Commit Server using an offset merge type and the Commit Server as the left input database so no change renumbering occurs in that server
$ perfmerge -t offset -o 10000 /commit/server/p4root /other/server/p4root /merged/server/p4root5. Against the post-merged server, create a checkpoint to use for reseeding the Edge Server
$ cd /merged/server/p4root $ p4d -r . -K "db.have,db.working,db.resolve,db.locks,db.revsh,db.workingx,db.resolvex" -z -jd commit-reseed-edge.ckp.gz6. On the Edge Server, replay the newly created post-merged Commit Server checkpoint and the Edge Server checkpoint from step 3 above into the edge P4ROOT
$ cd $P4ROOT $ rm -rf db.* $ p4d -r . -z -jr commit-reseed-edge.ckp.gz $ p4d -r . -jr edge.ckp7. Restart the Commit and Edge Server
It's a good idea to perform the following checks before performing the merge; in the end, it will save you time.
You should always, ALWAYS take a checkpoint and backup of both source databases before attempting a merge!
Perfmerge supports three different types of merge; they are all broadly similar and only differ in the way that changelists and jobs are renumbered as part of the merge.
The simple merge is time-based: changelists from both databases are sorted by submit time, and then renumbered, starting from 1.
Jobs are sorted by jobname and by the order of the databases on the command line (i.e. if a job exists in both databases, the job from the first (left) database comes before the job from the second (right) database).
In an append merge, the changelists and jobs from the second database on the command line are 'appended' to those in the first database. This has the advantage of preserving the changelist numbers and job numbers from the first database, and the disadvantage that changelists no longer have their temporal nature. i.e. 'p4 files //...@<change>' can no longer be assumed to represent the state of your repository at a point in time.
An offset merge is much like an append merge except that the changes and jobs from the second database are 'offset' from those in the first database. So, for example, if the first database had as its highest changelist change 14520, an offset merge might be used to append the second databases changes and jobs starting from 20000.
Perfmerge, at the end of the process, writes a "summary record" change to the file //depot/perfmerge/map.txt. This file is always written to the last used change ID in the merged server. The change counter will always point to the next free change ID after the summary record (if any).
The "-S" option provides three options for writing this record. These options are explained in the following table:
| Merge Type "-t" |
Merge Type Description | -S "offset" or no -S option specified |
-S "next" | -S "none" |
|---|---|---|---|---|
| simple | Time Based merge. Change IDs are "shuffled" into a time based sequence. |
A summary record is placed at the next change id which is a multiple of 1000 |
A summary record is placed at the next change id. |
No summary record is written. |
| append | Change IDs from the second server are appended to the first. |
A summary record is placed at the next change id which is a multiple of 1000 |
A summary record is placed at the next available change id. |
No summary record is written. |
| offset | Change IDs from second server are started at an offset after the end of the first. The offset is specified using the "-o" option. |
A summary record is placed at 2 * offset beyond the final change ID needed by the merge. |
A summary record is placed at the next available change id. |
No summary record is written. |
perfmerge -t type [-o offset] [-C n] [-cl | -cr] [-jl | -jr]
[-ul | -ur] [-v level ] [-V -h] [-S option]
left-P4ROOT right-P4ROOT output-P4ROOT
-t type - Type of merge to perform. Supported merge types are:
merge (simple, time-based merge)
append
offset
-o offset - For merge type offset, an additional offset value may
be specified.
-C n - Force case-sensitive (n=0), case-insensitive (n=1)
or 'hybrid' (n=2) ordering. Changing modes will
corrupt anything other than an empty database.
Note that all databases must use the same ordering.
-cl - Take server configuration data from left-P4ROOT.
-cr - Take server configuration data from right-P4ROOT.
-h - Help (this message)
-jl - Take job data only from left-P4ROOT. By default
job data is taken from both sources if they are
compatible.
-jr - Take job data only from right-P4ROOT. By default
job data is taken from both sources if they are
compatible.
-S option - Write Summary Change Options
'none' to disable writing summary change.
'next' to write the summary change at the
next available change ID.
'offset' to write the summary change at an offset
from the last change ID.
'offset' is the default behavior if -S is not used.
-sl - Take swarm data from the left.
-sr - Take swarm data from the right.
-sb - Merge swarm data from both sides.
-sz - Remove any swarm data.
-ul - Take conflicting user data from left-P4ROOT.
-ur - Take conflicting user data from right-P4ROOT.
-v level - Set verbosity level
-V - Print version and exit
After merging there is always some tidying up to do. The minimum you should do is:
Note: Perfmerge 'submits' a change to the target database as part of the merge process, the change contains a single file (//depot/perfmerge/map.txt), the contents of which describe the way that the changelists and jobs were renumbered. You can use the information in this file to test the results of the merge to your satisfaction. Perfmerge prints the changelist number for this change in its output at the end of the merge process.
$ mkdir /perforce/serverC $ perfmerge -t merge /perforce/serverA /perforce/serverB /perforce/serverC
This will merge the Perforce databases in /perforce/serverA and /perforce/serverB into the new Perforce database in /perforce/serverC. The merge will include jobs from both source databases providing the jobspecs are identical (otherwise it will fail).
When merging databases with incompatible jobspecs, one option is to only take job information from one of the databases, and completely ignore any job information in the othe. Like this:
$ mkdir /perforce/serverC $ perfmerge -t merge -jl /perforce/serverA /perforce/serverB /perforce/serverC
So in this case, the jobs will be taken from the left-hand source database (/perforce/serverA), and any jobs in /perforce/serverB will be ignored.
Say we want the jobs and changelists in serverA to be unchanged by the merge, but to merge in the contents of serverB. An append merge is just the ticket:
$ mkdir /perforce/serverC $ perfmerge -t append /perforce/serverA /perforce/serverB /perforce/serverC
Now let's suppose we want to be able to immediately see where merged changes came from when we look back at the history in the merged repository. In this case, an append merge doesn't quite fit the bill, and an offset merge is probably better:
$ mkdir /perforce/serverC $ perfmerge -t offset -o 20000 /perforce/serverA /perforce/serverB /perforce/serverC
This will merge serverB with serverA, leaving serverA's change and job numbers unchanged, and appending serverB's changes and jobs starting with change/job 20000 (which must be greater than any existing change/job in serverA).