Branch and Submit Benchmark Description The Perforce Server (P4D) synchronizes access to its metadata through the use of file locks on the db.* tables. For Perforce tasks that only need to read portions of the metadata, P4D takes read locks on only those db.* tables containing metadata needed by the task. If a task needs to update metadata within a db.* table, P4D takes a write lock on the db.* table. A read lock on a db.* table can be shared with other read locks on the same db.* table, but a write lock on a db.* table is exclusive of all other locks on that db.* table. In general, P4D minimizes the duration that a write lock is held on a db.* table. One notable exception that can result in P4D holding a write lock on a db.* table for an extended duration is the commit portion of a large changelist submission's dm-CommitSubmit phase. Since the commit portion must be atomic, P4D holds write locks on several important db.* tables for the duration of the commit portion of a changelist submission's dm-CommitSubmit phase. The write locks held block all other tasks that need access to the same tables. It is therefore important that the commit portion of a changelist submission's dm-CommitSubmit phase execute as fast as possible so that the write locks are released, making the db.* tables available for access to the waiting tasks. The metadata used in this benchmark can be the reference01 metadata or metadata that is more meaningful to a site. Using site-specific metadata can result in performance observations applicable to the site. Using the reference01 metadata will result in performance observations that can be compared to those at other sites. Using the reference01 metadata with the current release will require approximately 45GB of disk space in the P4ROOT directory and for each branch created, submitted, and obliterated by the benchmark, 308MB of disk space for the journal. Running the Benchmark It is strongly recommended that you first acquaint yourself with this benchmark on an isolated test machine before executing the benchmark on a production machine. This benchmark must be kept separate from any existing Perforce installations. Of particular note is that the value for the benchmark's P4ROOT definition must not be the same as an existing Perforce installation. The first portion of the branchsubmit.sh script defines the utilities (awk, cut, date, etc.) used in the script. If any of these utilities are not in your path, their definition can be changed to their absolute location, or to suitable alternative utilities. Of particular note are the definitions for p4 and p4d. Their definitions can be changed to use specific releases: p4=//p4 p4d=//p4d The next portion of the script defines filesystems and other variables used in the script. The definitions for the following will need changed to values that are suitable for your site: P4CLIENTdir P4ROOT P4JOURNALdir P4LOGdir ZCKP The script will create the P4CLIENTdir, P4ROOT, P4JOURNALdir, and P4LOGdir directories. The directories used for this benchmark must not be part of any existing Perforce installation. If the values for the P4CLIENTdir, P4ROOT, P4JOURNALdir, and P4LOGdir definitions are part of an existing Perforce installation, the existing installation can be corrupted. The value of the ZCKP definition must be the compressed checkpoint of the metadata that will be used in the benchmark. This is a compressed checkpoint of the reference01 metadata or site-specific metadata. The db.* files will be created using the compressed checkpoint during the script's "setup" phase. The BRANCHCMDS variable defines the integrate commands that will be executed to open the files for branch. This is typically a single integrate command, but can be multiple integrate commands to observe the performance behavior of submitting a larger number of files. For example, using the reference01 metadata, the default BRANCHCMDS definition will submit 70,000 files during each "runme" phase: BRANCHCMDS="integrate -v //depot/main/0... //depot/r36.0.0/0..." Using the reference01 metadata, the following BRANCHCMDS definition will submit 210,000 files during each "runme" phase: BRANCHCMDS="integrate -v //depot/main/0... //depot/r36.0.0/0... integrate -v //depot/main/1... //depot/r36.0.0/1... integrate -v //depot/main/2... //depot/r36.0.0/2..." (Note that each integrate command is on a separate line and that there is a leading double quote prior to the first integrate command and a trailing double quote after the last integrate command.) If using site-specific metadata, you might need to change the definitions for the following to values that are suitable for the site: BRANCHCMDS P4CLIENT P4USER Depending upon the ports already in use on the machine where the benchmark is executed, the value of the script's P4PORT definition might need to be changed. The value must not be the same as that used by another P4D on the machine. After the necessary changes have been made to the values of the applicable definitions within the script, the benchmark is then executed from the command line (no arguments are necessary): ./branchsubmit.sh When executed using no arguments, the script executes the "setup" phase, two instances of the "runme" phase, and then the "cleanup" phase. The "setup" phase creates the db.* files using the compressed checkpoint and creates a client that will be used in the benchmark. The "runme" phase branches and submits the branch, followed by an obliterate of the branched files and deletion of the changelist so that the "runme" phase can be executed more than once. The "cleanup" phase moves the P4D log to the directory from which the script is executed and deletes the db.* files and the journal used in the benchmark. Specific phases of the benchmark can be executed by the script. The script will execute each phase as specified by the arguments given to the script. For example, if executing the "setup" phase and three instances of the "runme" phase is desired, the script would be invoked as: ./branchsubmit.sh setup runme runme runme Interpreting the Results The interesting metrics reported by the benchmark's "runme" phase are the duration of the integrate's compute phase and the duration of the commit portion of the changelist submission's dm-CommitSubmit phase: Starting branchsubmit runme phase... Branching files... sleeping... executing... sleeping... done. compute phase duration: 7002 milliseconds files opened: 70000 Creating changelist... sleeping... executing... sleeping... done. Submitting changelist... sleeping... executing... sleeping... done. commit duration: 9997 milliseconds commit rate: 7002 files/second Obliterating changelist... sleeping... executing... sleeping... done. For both the "compute phase duration" and the "commit duration", smaller is better. For the "commit rate", larger is better. By default, the benchmark executes the "runme" phase twice. Since the first "runme" phase is executed immediately after the metadata has been populated from the checkpoint, the results of the first "runme" phase might not be realistic. Immediately after replaying a checkpoint, the metadata is tightly packed on the db.* pages. The first dm-CommitSubmit therefore incurs the penalty of numerous page splits, resulting in more db.* pages written than would otherwise be necessary. At the end of the first "runme" phase, the branched files are obliterated. After the files have been obliterated, there will be free space on some of the db.* pages. With free space on some of the db.* pages, the page split overhead incurred during the second "runme" phase's dm-CommitSubmit will be minimal. Though a site's workflow can affect the free space within the db.* tables, having some free space is probably more realistic than no free space at all. Therefore, the results of the second "runme" phase are probably more realistic. The state of the filesystem cache also has a significant effect on performance. This benchmark does not flush the filesystem cache, so the second "runme" phase is likely executed with a warm filesystem cache. In a production environment, the state of the filesystem cache immediately prior to executing an integrate command is dependent upon the site's workflow. But immediately prior to the dm-CommitSubmit, the filesystem cache is very likely warm, since prior phases of the changelist submission have already used the same metadata. Results observed in this benchmark using the reference01 metadata might not be representative of results observed in a production environment, though the hardware might be comparable. Many characteristics of the metadata can affect P4D performance. One such characteristic is the length of paths in the depot filenames. Recommended Actions The Perforce Knowledge Base contains several articles that make recommendations for improving the performance of your Perforce Server. One of these articles is: http://kb.perforce.com/?article=931 These recommendations apply to the Perforce commands used in the Branch and Submit Benchmark, and in general, to all Perforce commands.