Perforce Proxy
Perforce is built to handle distributed development in a wide range of network topologies. Where bandwidth to remote sites is limited, P4P, the Perforce Proxy, improves performance by mediating between Perforce applications and the versioning service to cache frequently transmitted file revisions. By intercepting requests for cached file revisions, P4P reduces demand on the Perforce service and the network over which it runs.
To improve performance obtained by multiple Perforce users accessing a shared Perforce repository across a WAN, configure P4P on the side of the network close to the users and configure the users to access the service through P4P; then configure P4P to access the master Perforce service. (On a LAN, you can also obtain performance improvements by setting up proxies to divert workload from the master server’s CPU and disks.)
The following diagram illustrates a typical P4P configuration:
In this configuration, file revisions requested by users at a remote development
site are fetched first from a central Perforce server (p4d
running on
central
) and transferred over a relatively slow WAN. Subsequent requests for
that same revision, however, are delivered from the Perforce Proxy, (p4p
running on outpost
), over the remote development site’s LAN, reducing both
network traffic across the WAN and CPU load on the central server.
System requirements
To use Perforce Proxy, you must have:
- A Perforce Server at Release 2002.2 or higher (2012.1 or higher to use SSL)
- Sufficient disk space on the proxy host to store a cache of file revisions
Installing P4P
In addition to the basic steps described next you might also want to do the following:
- Enable SSL support. See Enabling SSL support for more information.
- Defend against man-in-the-middle attacks. See Defending from man-in-the-middle attacks for more information.
UNIX
To install P4P on UNIX or Linux, do the following:
- Download the
p4p
executable to the machine on which you want to run the proxy. - Select a directory on this machine (
P4PCACHE
) in which to cache file revisions. - Select a port (
P4PORT
) on whichp4p
will listen for requests from Perforce applications. - Select the target Perforce server (
P4TARGET
) for which this proxy will cache.
Windows
Install P4P from the Windows installer’s custom/administrator installation dialog.
Running P4P
To run P4P, invoke the p4p
executable, configuring it with environment
variables or command-line options. Options you specify on the command line
override environment variable settings.
For example, the following command line starts a proxy that communicates with a
central Perforce server located on a host named central
, listening on port
1666.
$ p4p -p tcp64:[::]:1999 -t central:1666 -r /var/proxyroot
To use the proxy, Perforce applications connect to P4P on port 1999 on the
machine where the proxy runs. The proxy listens on both the IPv6 and IPv4
transports. P4P file revisions are stored under a directory named
/var/proxyroot
.
The Perforce proxy supports connectivity over IPv6 networks as well as IPv4. See the Helix Versioning Engine Administrator Guide: Fundamentals for more information.
Running P4P as a Windows service
To run P4P as a Windows service, either install P4P from the Windows installer,
or specify the -s
option when you invoke p4p.exe
, or rename the P4P
executable to p4ps.exe
.
To pass parameters to the P4Proxy service, set the P4POPTIONS
registry
variable using the p4 set
command. For example, if you normally run the
Proxy with the command:
C:\> p4p -p 1999 -t ssl:mainserver:1666
You can set the P4POPTIONS
variable for a Windows service named Perforce
Proxy
by setting the service parameters as follows:
C:\> p4 set -S "Perforce Proxy" P4POPTIONS="-p 1999 -t ssl:mainserver:1666"
When the "Perforce Proxy"
service starts, P4P listens for plaintext
connections on port 1999 and communicates with the Perforce Server via SSL at
ssl:mainserver:1666
.
P4P options
The following command-line options specific to the proxy are supported.
Proxy options:
Option | Meaning |
---|---|
|
Run as daemon - fork first, then run (UNIX only). |
|
Do not fork - run as a single-threaded server (UNIX only). |
|
Run for |
|
Run quietly; suppress startup messages. |
|
Do not compress data stream between the Perforce server to P4P. (This option reduces CPU load on the central server at the expense of slightly higher bandwidth consumption.) |
|
Run as a Windows service (Windows only). Running |
|
Disable cache fault coordination. The proxy maintains a table of concurrent sync operations, called When |
General options:
Option | Meaning |
---|---|
|
Display a help message. |
|
Display the version of the Perforce Proxy. |
|
Specify the directory where revisions are cached. Default is |
|
Specify the location of the log file. Default is |
|
Specify the port on which P4P will listen for requests from Perforce
applications. Default is |
|
Specify the port of the target Perforce server (that is, the Perforce server
for which P4P acts as a proxy). Default is |
|
Cache only those files that are larger than |
|
For proxy servers, authenticate as the specified |
|
Specifies server trace level. Debug messages are stored in the proxy server’s
log file. Debug messages from |
Certificate-handling options:
Option | Meaning |
---|---|
-Gc |
Generate SSL credentials files for the proxy: create a private key
( Requires that |
-Gf |
Display the fingerprint of the proxy’s public key, and exit. Administrators can communicate this fingerprint to end users, who can then use
the |
Proxy monitoring options:
Option | Meaning |
---|---|
|
List pending archive transfers |
|
List pending archive transfers, summarized |
|
Set the file status interval, in seconds. If not set, defaults to 10 seconds. |
|
|
|
Proxy monitoring interval, in seconds. If not set, defaults to 10 seconds. |
|
Show currently-active connections and their status. Requires |
Proxy archive cache options:
Option | Meaning |
---|---|
|
|
Administering P4P
The following sections describe the tasks involved in administering a proxy.
No backups required
You never need to back up the P4P cache directory.
If necessary, P4P reconstructs the cache based on Perforce server metadata.
Stopping P4P
P4P is effectively stateless; to stop P4P under UNIX, kill
the p4p
process with SIGTERM
or SIGKILL
. Under Windows, click End Process in the
Task Manager.
Upgrading P4P
After you have replaced the p4p
executable with the upgraded version,
you must also remove the pdb.lbr
and pdb.monitor
files (if they exist) from
the proxy root before you restart the upgraded proxy.
Enabling SSL support
To encrypt the connection between a Perforce Proxy and its end users, your proxy
must have a valid private key and certificate pair in the directory specified by
its P4SSLDIR
environment variable. Certificate and key generation and
management for the proxy works the same as it does for the Perforce Server. See
Enabling SSL support. The users' Perforce applications must be
configured to trust the fingerprint of the proxy.
To encrypt the connection between a Perforce Proxy and its upstream Perforce
service, your proxy installation must be configured to trust the fingerprint of
the upstream Perforce service. That is, the user that runs p4p
(typically a service user) must create a P4TRUST
file (using p4 trust
)
that recognizes the fingerprint of the upstream Perforce service.
For complete information about enabling SSL for the proxy, see:
http://answers.perforce.com/articles/KB/2596
Defending from man-in-the-middle attacks
You can use the net.mimcheck
configurable to enable checks for possible
interception or modification of data. These settings are pertinent for proxy
administration:
- A value of 3 checks connections from clients, proxies, and brokers for TCP forwarding.
- A value of 5 requires that proxies, brokers, and all Perforce intermediate servers have valid logged-in service users associated with them. This allows administrators to prevent unauthorized proxies and services from being used.
You must restart the server after changing the value of this configurable. For more information about this configurable, see the "Configurables" appendix in P4 Command Reference.
Localizing P4P
If your Perforce server has localized error messages (see "Localizing server
error messages" in Helix Versioning Engine Administrator Guide: Fundamentals), you can localize your proxy’s error message
output by shutting down the proxy, copying the server’s db.message
file into
the proxy root, and restarting the proxy.
Managing disk space consumption
P4P caches file revisions in its cache directory. These revisions accumulate until you delete them. P4P does not delete its cached files or otherwise manage its consumption of disk space.
Warning
If you do not delete cached files, you will eventually run out of disk space. To recover disk space, remove files under the proxy’s root.
You do not need to stop the proxy to delete its cached files or the pdb.lbr
file.
If you delete files from the cache without stopping the proxy, you must also
delete the pdb.lbr
file at the proxy’s root directory. (The proxy uses the
pdb.lbr
file to keep track of which files are scheduled for transfer, so that
if multiple users simultaneously request the same file, only one copy of the
file is transferred.)
Determining if your Perforce applications are using the proxy
If your Perforce application is using the proxy, the proxy’s version information
appears in the output of p4 info
.
For example, if a Perforce service is hosted at ssl:central:1666
and you
direct your Perforce application to a Perforce Proxy hosted at outpost:1999
,
the output of p4 info
resembles the following:
$export P4PORT=tcp:outpost:1999
$p4 info
User name: p4adm Client name: admin-temp Client host: remotesite22 Client root: /home/p4adm/tmp Current directory: /home/p4adm/tmp Client address: 192.168.0.123 Server address: central:1666 Server root: /usr/depot/p4d Server date: 2012/03/28 15:03:05 -0700 PDT Server uptime: 752:41:23 Server version: P4D/FREEBSD4/2012.1/406375 (2012/01/25) Server encryption: encrypted Proxy version: P4P/SOLARIS26/2012.1/406884 (2012/01/25) Server license: P4 Admin <p4adm> 20 users (expires 2013/01/01) Server license-ip: 10.0.0.2 Case handling: sensitive
P4P and protections
To apply the IP address of a Perforce Proxy user’s workstation against the
protections table, prepend the string proxy-
to the workstation’s IP address.
For instance, consider an organization with a remote development site with
workstations on a subnet of 192.168.10.0/24
. The organization also has a
central office where local development takes place; the central office exists on
the 10.0.0.0/8
subnet. A Perforce service resides in the 10.0.0.0/8
subnet,
and a Perforce Proxy resides in the 192.168.10.0/24
subnet. Users at the
remote site belong to the group remotedev
, and occasionally visit the central
office. Each subnet also has a corresponding set of IPv6 addresses.
To ensure that members of the remotedev
group use the proxy while working at
the remote site, but do not use the proxy when visiting the local site, add the
following lines to your protections table:
list group remotedev 192.168.10.0/24 -//... list group remotedev [2001:db8:16:81::]/48 -//... write group remotedev proxy-192.168.10.0/24 //... write group remotedev proxy-[2001:db8:16:81::]/48 //... list group remotedev proxy-10.0.0.0/8 -//... list group remotedev proxy-[2001:db8:1008::]/32 -//... write group remotedev 10.0.0.0/8 //... write group remotedev proxy-[2001:db8:1008::]/32 //...
The first line denies list
access to all users in the remotedev
group if
they attempt to access Perforce without using the proxy from their workstations
in the 192.168.10.0/24
subnet. The second line denies access in identical
fashion when access is attempted from the IPV6 [2001:db8:16:81::]/48
subnet.
The third line grants write
access to all users in the remotedev
group if
they are using a Perforce Proxy server and are working from the
192.168.10.0/24
subnet. Users of workstations at the remote site must use the
proxy. (The proxy server itself does not have to be in this subnet, for example,
it could be at 192.168.20.0
.) The fourth line denies access in identical
fashion when access is attempted from the IPV6 [2001:db8:16:81::]/48
subnet.
Similarly, the fifth and sixth lines deny list
access to remotedev
users
when they attempt to use the proxy from workstations on the central office’s
subnets (10.0.0.0/8
and [2001:db8:1008::]/32
). The seventh and eighth
lines grant write access to remotedev
users who access the Perforce server
directly from workstations on the central office’s subnets. When visiting the
local site, users from the remotedev
group must access the Perforce server
directly.
When the Perforce service evaluates protections table entries, the
dm.proxy.protects
configurable is also evaluated.
dm.proxy.protects
defaults to 1
, which causes the proxy-
prefix to be
prepended to all client host addresses that connect via an intermediary (proxy,
broker, replica, or edge server), indicating that the connection is not direct.
Setting dm.proxy.protects
to 0
removes the proxy-
prefix and allows you to
write a single set of protection entries that apply both to directly-connected
clients as well as to those that connect via an intermediary. This is more
convenient but less secure if it matters that a connection is made using an
intermediary. If you use this setting, all intermediaries must be at release
2012.1 or higher.
Determining if specific files are being delivered from the proxy
Use the -Zproxyverbose
option with p4
to display messages indicating
whether file revisions are coming from the proxy (p4p
) or the central
server (p4d
). For example:
$ p4 -Zproxyverbose sync noncached.txt
//depot/main/noncached.txt - refreshing /home/p4adm/tmp/noncached.txt
$ p4 -Zproxyverbose sync cached.txt
//depot/main/cached.txt - refreshing /home/p4adm/tmp/cached.txt
File /home/p4adm/tmp/cached.txt delivered from proxy server
Case-sensitivity issues and the proxy
If you are running the proxy on a case-sensitive platform such as UNIX, and your
users are submitting files from case-insensitive platforms (such as Windows),
the default behavior of the proxy is to fold case; that is, FILE.TXT
can
overwrite File.txt
or file.txt
.
In the case of text files and source code, the performance impact of this
behavior is negligible. If, however, you are dealing with large binaries such as
.ISO
images or .VOB
video objects, there can be performance issues
associated with this behavior.)
lbr.proxy.case | Behavior |
---|---|
|
Proxy folds case; all files with the same name are assumed to be the same file, regardless of case. |
|
Proxy folds case if, and only if, the upstream server is case-insensitive (that is, if the upstream server is on Windows) |
|
Proxy never folds case. |
After any change to lbr.proxy.case
, you must clear the cache before restarting
the proxy.
Maximizing performance improvement
Reducing server CPU usage by disabling file compression
By default, P4P compresses communication between itself and the Perforce
versioning service, imposing additional overhead on the service. To disable
compression, specify the -c
option when you invoke p4p
. This option is
particularly effective if you have excess network and disk capacity and are
storing large numbers of binary file revisions in the depot, because the proxy
(rather than the upstream versioning service) decompresses the binary files from
its cache before sending them to Perforce users.
Network topologies versus P4P
If network bandwidth on the same subnet as the Perforce service is nearly saturated, deploying proxy servers on the same subnet will not likely result in a performance improvement. Instead, deploy the proxies on the other side of a router so that the traffic from end users to the proxy is isolated to a subnet separate from the subnet containing the Perforce service.
For example:
Deploying an additional proxy on a subnet when network bandwidth on the subnet is nearly saturated will not likely result in a performance improvement. Instead, split the subnet into multiple subnets and deploy a proxy in each resulting subnet.
In the illustrated configuration, a server room houses a company’s Perforce
service (p4d
), a network storage device (NAS), and a database server
(RDBMS). The server room’s network segment is saturated by heavy loads placed on
it by a sales force constantly querying a database for live updates, and by
developers and graphic artists frequently accessing large files versioned by
Perforce.
Deploying two instances of the Perforce proxy (one on the developers' subnet, and one on the graphic artists' subnet) enables all three groups to benefit from improved performance due to decreased use on the server room’s network segment.
Preloading the cache directory for optimal initial performance
P4P stores file revisions only when one of its users submits a new revision to the depot or requests an existing revision from the depot. That is, file revisions are not prefetched. Performance gains from P4P occur only after file revisions are cached.
After starting P4P, you can effectively prefetch the cache directory by creating
a dedicated client workspace and syncing it to the head revision. All other
users who subsequently connect to the proxy immediately obtain the performance
improvements provided by P4P. For example, a development site located in Asia
with a P4P server targeting a Perforce server in North America can preload its
cache directory by using an automated job that runs a p4 sync
against
the entire Perforce depot after most work at the North American site has been
completed, but before its own developers arrive for work.
By default, p4 sync
writes files to the client workspace. If you have a
dedicated client workspace that you use to prefetch files for the proxy,
however, this step is redundant. If this machine has slower I/O performance than
the machine running the Perforce Proxy, it can also be time-consuming.
To preload the proxy’s cache without the redundant step of also writing the
files to the client workspace, use the -Zproxyload
option when syncing. For
example:
$ export P4CLIENT=prefetch
$ p4 sync //depot/main/written.txt
//depot/main/written.txt - refreshing /home/prefetch/main/written.txt
$ p4 -Zproxyload sync //depot/main/nonwritten.txt
//depot/main/nonwritten.txt - file(s) up-to-date.
Both files are now cached, but nonwritten.txt
is never written to the the
prefetch
client workspace. When prefetching the entire depot, the time savings
can be considerable.
Distributing disk space consumption
P4P stores revisions as if there were only one depot tree. If this approach stores too much file data onto one filesystem, you can use symbolic links to spread the revisions across multiple filesystems.
For instance, if the P4P cache root is /disk1/proxy
, and the Perforce server
it supports has two depots named //depot
and //released
, you can split data
across disks, storing //depot
on disk1
and //released
on disk2
as
follows:
$ mkdir /disk2/proxy/released
$ cd /disk1/proxy
$ ln -s /disk2/proxy/released released
The symbolic link means that when P4P attempts to cache files in the
//released
depot to /disk1/proxy/released
, the files are stored on
/disk2/proxy/released
.