32-bit operating systems might not be able to address large amounts of physical memory, which can restrict the effective size of the filesystem cache. The various 64-bit operating systems each have their own performance characteristics that can favor a particular Perforce workload. In general, Linux distributions using later Linux 2.6 64-bit kernels have good performance characteristics for most Perforce workloads.

For I/O requests that must be satisfied from beyond the filesystem cache, there might be several improvements possible for the I/O subsystem. The storage subsystem containing the db.* files should have a memory cache; maximizing the storage subsystem’s memory cache is also a good recommendation. For best performance, write-back caching should be enabled, which of course requires that the storage subsystem’s memory have battery backup power. I/O latency to the logical drive where the db.\* files are located should be minimized, including the rotational latency of the physical drives themselves. Minimizing I/O latency might require direct connections between the host and the storage subsystem, and usually requires physical drives with the fastest rotational speed (such as 15K RPM).

RAID 1+0 (or RAID 10) is usually the better performing RAID configuration, and is recommended for the logical drive where the db.* files are located. The number of physical drives in the logical drive can also have an affect on *p4d* performance. Generally, performance improves as the number of physical drives in the logical drive increases. For a given amount of disk space required, better performance might result from using more smaller-capacity physical drives. The stripe size for the logical drive can also affect performance; the optimal stripe size might be dependent upon the number of physical drives in the logical drive.

Hardware-based RAID implementations (that is, RAID logic that is not implemented as software running on the host) usually have good performance characteristics. Software-based RAID implementations can require CPU cycles that might otherwise be needed for p4d processes. Therefore, software-based RAID implementations should be avoided.

Filesystem performance is an important component of operating system performance. The various operating systems usually offer several filesystems, each with their own performance characteristics that can favor a particular Perforce workload. For best p4d performance, the db.* files should be located on a high-performance filesystem. In general, the XFS filesystem has good performance characteristics for most Perforce workloads. The XFS filesystem is available on several operating systems, including Linux distributions using later Linux 2.6 64-bit kernels.

Reading pages into a cache in anticipation of being requested is an optimization that is often implemented within various I/O subsystem components. This optimization is commonly known as "read-ahead". In some implementations, read-ahead can be tuned, which might result in better performance. But tuning read-ahead can be a bit of an art. For example, increasing the read-ahead size might result in better performance for operations requiring mostly sequential reads. But the same increased read-ahead size applied consistently during random reads might unnecessarily discard previously-cached data that might have satisfied subsequent requests.

CPU resource consumption can be adversely affected by compression, lockless reads, or a badly designed protections table. In general, there is a trade-off between speed and the number of cores. A minimum of 2.4 GHZ and 8 cores is recommended. With greater speed, fewer cores will do: for example, a 3.2 GHZ and 4-core processor will also work.

Faster processors and memory in the machine where p4d executes might result in faster execution of p4d commands. Since portions of some commands acquire and hold resources that might block other commands, it is important that these portions of the commands execute as fast as possible. For example, most p4d commands have a compute phase, during which shared locks are acquired and held on some of the db.* files. A shared lock on a db.\* file blocks an operation that writes to the same db.\* file. If the data needed for a command’s compute phase is cached within the operating system’s filesystem cache, only the processor and memory speed constrains the compute phase.

If you are using lockless reads, CPU speed is not as critical, but can still be helpful for good performance. Since some readers will no longer block a writer (and a writer will no longer block some readers), speeding commands through the server might not be as critical from a concurrency point of view. And since more commands might now run concurrently through the Perforce Server, more CPU cores might be better utilized.

The complexity of the site’s protections table and of client views can affect CPU requirements. You can monitor CPU utilization using OS utilities such as top (on Linux and Unix) and perfmon (on Windows). Installations with high CPU utilization on the machine where p4d executes that are already using faster processors might need more processors and/or processors with more cores while maintaining the speed of the processors.

Some processors and operating systems support dynamic frequency scaling, which allows the processor to vary power consumption by dynamically adjusting the processor voltage and core frequency. As more demand is placed on the processor, the voltage and core frequency increase. Until the processor is ramped up to full speed, p4d performance might be impacted. Although the power-saving capability of the dynamic frequency scaling feature is useful for mobile computers, it is not recommended for the machine where p4d executes.

Examples of dynamic frequency scaling include the following:

Both features are supported on Linux (and enabled by default in some SuSE distributions), Windows, and Mac OS X platforms. If this feature is enabled on the machine where p4d executes, we recommend disabling it. In some Linux distributions, such as SuSE, this feature can be disabled by setting the "powersaved" service to "off".

You might be able to determine the current speed of the processors on your computer. On Linux, the current speed of each core is reported on the "cpu MHz" line in the output from the cat /proc/cpuinfo OS command.

Server performance is highly dependent upon having sufficient memory. Two bottlenecks are relevant. The first bottleneck can be avoided by ensuring that the server doesn’t page when it runs large queries, and the second by ensuring that the db.rev table (or at least as much of it as practical) can be cached in main memory:

Thus, if there is 1.5 kilobytes of RAM available per file, or 150 MB for 100,000 files, the server does not page, even when performing operations involving all files. It is still possible that multiple large operations can be performed simultaneously and thus require more memory to avoid paging. On the other hand, the vast majority of operations involve only a small subset of files.

One way to determine if you have allocated sufficient memory is to look at the physical read rate on the device that contains only the database files. This read rate should be trivial.

Perforce can run over any TCP/IP network. For remote users or distributed configurations, Perforce offers options like proxies and the commit/edge architecture that can enhance performance over a WAN. Compression in the network layer can also help.

Perforce uses a TCP/IP connection for each client interaction with the server. The server’s port address is defined by P4PORT, but the TCP/IP implementation picks a client port number. After the command completes and the connection is closed, the port is left in TIME_WAIT state for two minutes. Although the port number ranges from 1025 to 32767, generally only a few hundred or thousand can be in use simultaneously. It is therefore possible to occupy all available ports by invoking a Perforce command many times in rapid succession, such as with a script.

By default, idle connections are not kept alive. If your network silently drops idle connections, this behavior may cause unexpected connectivity issues. (Consider a p4 pull thread that transfers data between a master server and a replica at a remote site; depending on each site’s respective business hours and user workloads, such a connection could be idle for several hours per day.) Four configurables are available to manage the state of idle connections.

If your network configuration requires keepalive packets, consider setting net.keepalive.idle to a suitably long value, for example 3,600 seconds (1 hour), and an interval measured in tens of minutes.

For recoverability, the live journal should not be on the same physical device that contains the db.* files. Separating the live journal and the db.\* files also improves performance. During operations that write to the db.* files, entries are written to the live journal as records are written to the db.\* files. If the live journal and the db.* files are on the same physical device, the I/O throughput to the db.\* files is degraded. For best performance, the live journal should be on a separate storage subsystem connected to a separate host adapter. The live journal should be on a logical drive and filesystem that is optimized for sequential writes.

The versioned files should be located on a separate logical drive than the logical drives where the db.* files and the live journal are located. For best performance, the logical drive where the versioned files are located should be on a separate storage subsystem connected to a separate host adapter. Since the versioned files typically require significantly more disk space and the I/O throughput is not as critical as for the db.\* files, a more economical RAID configuration, such as RAID 5, can be used for the logical drive where the versioned files are located.

Perforce usage can affect performance. There are several usage patterns that can have a direct effect on performance. Since the depot filenames are the leading portion of the key in several important db.\* files (db.rev, db.revhx, and db.integed are among the more notable), the length of paths in the depot filenames have a direct effect on performance. As the length of paths increase, performance decreases. It is therefore prudent to discourage the use of overly-descriptive paths in the depot filenames.

The development methodology can also have a direct effect on performance. If the development methodology calls for frequent creation of full branches (perhaps branching for each bug fix), then the amount of metadata rapidly increases, resulting in more levels within the db.* file B-trees. As the number of levels increase, more key comparisons and I/O requests are required to traverse to the leaf pages, which will impact performance. Creating full branches also requires more metadata read and written; the additional metadata read and written might affect the filesystem cache to the detriment of other Perforce tasks. Rather than frequent creation of full branches, it might be prudent to branch only those files needed for each bug fix, or consider a development methodology in which multiple bug fixes can occur on the same branch.

Build automation scripts, which routinely create, sync, and tear down clients, may fragment the db.have table over time. To avoid this, you can specify the type readonly for these clients. Such clients cannot add, delete, edit, integrate, or submit files, but this should not be an issue in build scripts.

A readonly client is assigned its own personal db.have database table, and the location of this table is specified using the client.readonly.dir configurable.

To set up a read-only client:

You can configure the server to transfer files in parallel for submit and sync processing. Parallel processing is most effective with long-haul, high latency networks or with other network configuration that prevents the use of available bandwidth with a single TCP flow. Parallel processing might also be appropriate when working with large compressed binary files, where the client must perform substantial work to decompress the file.

For more information see the p4 submit command and the p4 sync command in P4 Command Reference.

When peeking is enabled, the following commands run lockless:

The following commands run partially lockless; in most cases these commands will operate lock-free, but lockless operation is not guaranteed:

You can override the db.peeking setting on a per-command basis by using the -Zpeeking= flag followed by your preferred value. For example, to disable peeking for one command, run the following command:

$ p4 -Zpeeking=1 fstat

and compare the results with:

$ p4 -Zpeeking=2 fstat

To determine whether read locks are impacting performance (and the extent to which enabling lockless reads has improved performance), you can examine the server logs, or you can use the -Ztrack flag to output, for any given command, the lines that would be written to the P4LOG. For example:

$ p4 -Zpeeking=1 -Ztrack sync

produces output for 11 database tables. The relevant lines here are those that refer to "locks read/write".

...
--- db.counters
---   pages in+out+cached 3+0+2
---   locks read/write 1/0 rows get+pos+scan put+del 1+0+0 0+0
--- db.user
---   pages in+out+cached 3+0+2
---   locks read/write 1/0 rows get+pos+scan put+del 1+0+0 0+0
...

The 1 appearing in ("locks read/write 1/0") every table’s locking results shows one read lock taken per table. By contrast, the diagnostic output from:

$ p4 -Zpeeking=2 -Ztrack sync

...
--- db.counters
---   pages in+out+cached 3+0+2
---   locks read/write 0/0 rows get+pos+scan put+del 1+0+0 0+0
...

shows that the sync operation completed without any read or write locks required on db.counters (if you try it yourself, on many other tables); when peeking is enabled, many commands will show read/write 0/0 locks (or at least, fewer locks) taken.

A single Perforce instance can detect and ignore inadvertent attempts to override db.peeking that would change table locking order and risk deadlock. (For example, if you attempt to use db.peeking=3 on a server for which peeking is disabled by having db.peeking set to 0 (or unset), the service ignores the attempt altogether and the command proceeds with the old behavior.

In the case of "side-track servers" described in the following Knowledge Base article:

http://answers.perforce.com/articles/KB_Article/Setting-Up-a-Side-track-Server

this protection is not available.

Try setting P4PORT to the service’s IP address instead of its hostname. For example, instead of using:

P4PORT=host.domain:1666

try using:

P4PORT=1.2.3.4:1666

with your site-specific IP address and port number.

On most systems, you can determine the IP address of a host by invoking:

$ ping hostname

If p4 info responds immediately when you use the IP address, but not when you use the hostname, the problem is likely related to DNS.

In some cases, p4 commands on Windows can result in a delayed response if they use unquoted file patterns with a combination of depot syntax and wildcards, such as:

$ p4 files //depot/*

You can prevent the delay by putting double quotes around the file pattern, like this:

$ p4 files "//depot/*"

The cause of the problem is the p4 command’s use of a Windows function to expand wildcards. When quotes are not used, the function interprets //depot as a networked computer path and spends time in a futile search for a machine named depot.

On Windows, the %SystemRoot%\system32\drivers\etc\hosts file can be used to hardcode IP address-hostname pairs. You might be able to work around DNS problems by adding entries to this file. The corresponding UNIX file is /etc/hosts.

If none of the above diagnostic steps explains the sluggish response time, it’s possible that the p4 executable itself is on a networked file system that is performing very poorly. To check this, try running:

$ p4 -V

This merely prints out the version information, without attempting any network access. If you get a slow response, network access to the p4 executable itself might be the problem. Copying or downloading a copy of p4 onto a local filesystem should improve response times.

To set a hard upper bound on how long a connection is willing to wait on any single network read or write, set the net.maxwait configurable to the number of seconds to wait before disconnecting with a network error. Users working over unreliable connections can set net.maxwait value either in their P4CONFIG files, or use -vnet.maxwait=t on a per-command basis, where t is the number of seconds to wait before timing out.

It is useful to combine net.maxwait with the -rN global option, where N is the number of times to attempt reconnection in the event that the network times out. For example:

$ p4 -r3 -vnet.maxwait=60 sync

attempts to sync the user’s workspace, making up to three attempts to resume the sync if interrupted. The command fails after the third 60-second timeout.

Because the format of the output of a command that times out and is restarted cannot be guaranteed (for example, if network connectivity is broken in the middle of a line of output), avoid the use of -r on any command that reads from standard input. For example, the behavior of the following command, which reads a list of files from stdin and passes it to p4 add, can result in the attempted addition of "half a filename" to the depot.

$ find . -print | p4 -x - -r3 add

To prevent this from happening (for example, if adding a large number of files over a very unreliable connection), consider an approach like the following:

$ find directoryname -type f -exec p4 -r5 -vmax.netwait=60 add {} \;

All files (-type f) in directoryname are found, and added one at a time, by invoking the command "p4 -r5 -vmax.netwait=60 add" for each file individually.

After all files have been added, assign the changelist a changelist number with p4 change, and submit the numbered atomically with:

$ p4 -r5 -vmax.netwait=60 submit -c changenum

If connectivity is interrupted, the numbered changelist submission is resumed.

The following "loose" view is trivial to set up but could invite trouble on a very large depot:

//depot/...        //workspace/...

In the loose view, the entire depot was mapped into the client workspace; for most users, this can be "tightened" considerably. The following view, for example, is restricted to specific areas of the depot:

//depot/main/srv/devA/...          //workspace/main/srv/devA/...
//depot/main/drv/lport/...         //workspace/main/dvr/lport/...
//depot/rel2.0/srv/devA/bin/...    //workspace/rel2.0/srv/devA/bin/...
//depot/qa/s6test/dvr/...          //workspace/qa/s6test/dvr/...

Client views, in particular, but also branch views and label views, should also be set up to give users just enough scope to do the work they need to do.

Client, branch, and label views are set by a Perforce administrator or by individual users with the p4 client, p4 branch, and p4 label commands, respectively.

Two of the techniques for script optimization (described in Using branch views and Using a temporary client workspace) rely on similar techniques. By limiting the size of the view available to a command, fewer commands need to be run, and when run, the commands require fewer resources.

Protections (see Authorizing access) are actually another type of Perforce view. Protections are set with the p4 protect command and control which depot files can be affected by commands run by users.

Unlike client, branch, and label views, however, the views used by protections can be set only by Perforce superusers. (Protections also control read and write permission to depot files, but the permission levels themselves have no impact on server performance.) By assigning protections in Perforce, a Perforce superuser can effectively limit the size of a user’s view, even if the user is using "loose" client specifications.

Protections can be assigned to either users or groups. For example:

write     user       sam           *     //depot/admin/...
write     group      rocketdev     *     //depot/rocket/main/...
write     group      rocketrel2    *     //depot/rocket/rel2.0/...

Perforce groups are created by superusers with the p4 group command. Not only do they make it easier to assign protections, they also provide useful fail-safe mechanisms in the form of maxresults and maxscanrows, described in the next section.

Each Perforce group has an associated maxresults, maxscanrows, and maxlocktime value. The default for each is unset, but a superuser can use p4 group to limit it for any given group.

MaxResults prevents the server from using excessive memory by limiting the amount of data buffered during command execution. Users in limited groups are unable to run any commands that buffer more database rows than the group’s MaxResults limit. (For most sites, MaxResults should be larger than the largest number of files anticipated in any one user’s individual client workspace.)

Like MaxResults, MaxScanRows prevents certain user commands from placing excessive demands on the server. (Typically, the number of rows scanned in a single operation is roughly equal to MaxResults multiplied by the average number of revisions per file in the depot.)

Finally, MaxLockTime is used to prevent certain commands from locking the database for prolonged periods of time. Set MaxLockTime to the number of milliseconds for the longest permissible database lock.

To set these limits, fill in the appropriate fields in the p4 group form. If a user is listed in multiple groups, the highest of the MaxResults (or MaxScanRows, or MaxLockTime) limits (including unlimited, but not including the default unset setting) for those groups is taken as the user’s MaxResults (or MaxScanRows, or MaxLockTime) value.

Group:        rocketdev
MaxResults:   20000
MaxScanRows:  100000
MaxLockTime:  30000
Timeout:      43200
Subgroups:
Owners:
Users:
        bill
        ruth
        sandy

$ p4 sync

$ p4 sync //depot/projA/...
$ p4 sync //depot/projB/...

$ p4 filelog //depot/projA/...

To remove any limits on the number of result lines processed (or database rows scanned, or milliseconds of database locking time) for a particular group, set the MaxResults or MaxScanRows, or MaxLockTime value for that group to unlimited.

Because these limitations can make life difficult for your users, do not use them unless you find that certain operations are slowing down your server. Because some Perforce applications can perform large operations, you should typically set MaxResults no smaller than 10,000, set MaxScanRows no smaller than 50,000, and MaxLockTime to somewhere within the 1,000-30,000 (1-30 second) range.

For more information, including a comparison of Perforce commands and the number of files they affect, type:

$ p4 help maxresults
$ p4 help maxscanrows
$ p4 help maxlocktime

from the command line.

If monitoring is enabled (p4 configure set monitor=1 or higher), you can set the server.maxcommands configurable to limit the number of simultaneous command requests that the service will attempt to handle.

Ideally, this value should be set low enough to detect a runaway script or denial of service attack before the underlying hardware resources are exhausted, yet high enough to maintain a substantial margin of safety between the typical average number of connections and your site’s peak activity.

If P4LOG is set, the server log will contain lines of the form:

Server is now using nnn active threads.

You can use the server log to determine what levels of activity are typical for your site. As a general guideline, set server.maxcommands to at least 200-500% of your anticipated peak activity.

Over time, a Perforce server accumulates metadata associated with old projects that are no longer in active development. On large sites, reducing the working set of data, (particularly that stored in the db.have and db.labels tables) can significantly improve performance.

Depot:       unload
Type:        unload
Map:         unloaded/...

$ p4 unload -c oldworkspace
$ p4 unload -l oldlabel

The Perforce Command-Line Client, p4, supports the scripting of any command that can be run interactively. The Perforce server can process commands far faster than users can issue them, so in an all-interactive environment, response time is excellent. However, p4 commands issued by scripts — triggers, or command wrappers, for example — can cause performance problems if you haven’t paid attention to their efficiency. This is not because p4 commands are inherently inefficient, but because the way one invokes p4 as an interactive user isn’t necessarily suitable for repeated iterations.

This section points out some common efficiency problems and solutions.

for i in p4 diff2 path1/... path2/...
do
    [process diff output]
done

for i in p4 files path1/...
do
    p4 diff2 path1/$i path2/$i
    [process diff output]
done

for components in header1 header2 header3
do
    p4 edit ${component}.h
done

for components in header1 header2 header3
do
    echo ${component}.h >> LISTFILE
done
p4 -x LISTFILE edit

$ p4 diff2 pathA/src/...   pathB/src/...
$ p4 diff2 pathA/tests/... pathB/tests/...
$ p4 diff2 pathA/doc/...   pathB/doc/...

Branch:        pathA-pathB
View:
        pathA/src/...      pathB/src/...
        pathA/tests/...    pathB/tests/...
        pathA/doc/...      pathB/doc/...

$ p4 diff2 -b pathA-pathB

$ p4 files path/...@label | egrep "path/f1.h|path/f2.h|path/f3.h"

$ p4 files path/f1.h@label path/f1.h@label path/f3.h@label

$ p4 files path/f1.h@label
$ p4 files path/f2.h@label
$ p4 files path/f3.h@label

$ p4 sync pathA/src/...@label
$ p4 sync pathB/tests/...@label
$ p4 sync pathC/doc/...@label

Client:        XY-temp
View:
        pathA/src/...      //XY-temp/pathA/src/...
        pathB/tests/...    //XY-temp/pathB/tests/...
        pathC/doc/...      //XY-temp/pathC/doc/...

$ p4 -c XY-temp sync @label

There are cases where compression is automatically handled:

The Perforce server has many configurables that may be changed for performance purposes.

A complete list of configurables may be found by running p4 help configurables.