Streams
This chapter describes how to configure streams, how to propagate changes between them, and how to update them.
Note
If you are an existing user of Helix branches and would like to use streams instead, see the Streams Migration Guide.
Configure a stream
To configure a stream you edit its associated stream spec. A stream spec names
a path in a stream depot to be treated as a stream. A spec defines the stream’s
location, its type, its parent stream, the files in its view, and other
configurable behaviors. It is created when you create a stream with the p4
stream command. You can update the spec’s entries — as described in
Update streams — to change the stream’s characteristics.
The following is a sample stream spec:
$ p4 stream -o //Acme/dev
# A Perforce Stream Specification.
#
# Use *'p4 help stream'* to see more about stream specifications and command.
Stream: //Acme/dev
Update: 2015/02/06 10:57:04
Access: 2015/02/06 10:57:04
Owner: bruno
Name: //Acme/dev
Parent: //Acme/main
Type: development
Options: allsubmit unlocked toparent fromparent mergeany
Description:
Our primary development stream for the project.
Paths:
share ...
import boost/... //3rd_party/boost/1.53.0/artifacts/original/...
import boost/lib/linux26x86_64/... //3rd_party/boost/1.53.0/artifacts/original/lib/linuxx86_64/gcc44libc212/...
import boost/lib/linux26x86/... //3rd_party/boost/1.53.0/artifacts/original/lib/linuxx86/gcc44libc212/...
import protobuf/... //3rd_party/protobuf/2.4.1/artifacts/patch-1/...
import gtest/... //3rd_party/gtest/1.7.0/artifacts/original/...
import icu/... //3rd_party/icu/53.1/artifacts/original/...
import p4-bin/lib.ntx64/vs11/p4api_vs2012_dyn.zip //builds/p15.1/p4-bin/bin.ntx64/p4api_vs2012_dyn.zip
import p4/... //depot/p15.1/p4/...
exclude p4/lbr/...
exclude p4/server/...
Remapped:
p4/doc/... p4/relnotes/...
Ignored:
.../~tmp.txtThe following table describes the stream spec in more detail:
| Entry | Meaning |
|---|---|
|
|
The Error in stream specification. Stream |
|
|
The date the stream specification was last changed. |
|
|
The date the specification was originally created. |
|
|
The user or group who has specific and unique permissions to access to this stream. |
|
|
An alternate name of the stream, for use in display outputs. Defaults to the streamname portion of the stream path. |
|
|
The parent of this stream. Can be |
|
|
Type of stream provides clues for commands run between stream and parent. The
five types include |
|
|
A short description of the stream (optional). |
|
|
Stream Options: |
|
|
Identify paths in the stream and how they are to be generated in resulting
workspace views of this stream. Path types are
NoteFiles don’t actually have to be branched to appear in a stream. Instead, they can be imported from the parent stream or from other streams in the system. |
|
|
Remap a stream path in the resulting workspace view. |
|
|
Ignore a stream path in the resulting workspace view. Note that Perforce
recommends that you use |
More on options
The following table summarizes the meaning of each of the options available in the stream spec:
| Option | Meaning |
|---|---|
|
|
All users can submit changes to the stream. |
|
|
Only the stream owner can submit changes to the stream. |
|
|
The stream spec cannot be deleted and only the stream owner can modify it. |
|
|
All users can edit or delete the stream spec. |
|
|
Merges from the stream to its parent are expected. |
|
|
Merges from the stream to the parent are not expected. |
|
|
Merges to the stream from the parent are expected. |
|
|
Merges to the stream from the parent are not expected. |
|
|
Enforces the best practice of merge down, copy up. |
|
|
Allows you to merge the stream’s content both up and down. |
This section discusses some key concepts related to streams.
Stream types
You assign stream types according to the stream’s expected usage, stability and flow of change:
- Development streams are used for code that changes frequently; they they enable you to experiment without destabilizing the mainline stream.
- Mainline streams contain code that changes somewhat frequently, but is more stable than code in development streams.
- Release streams contain the most stable code, as this is the code closest to being released. Release streams enable you to finalize existing features while working on new features in the mainline.
There is also a virtual stream type and a task stream type. See Task streams and Virtual streams, respectively.
On a scale of stability, a development stream is considered less stable than its mainline stream parent, while a release stream is considered more stable than its mainline stream parent. Change is expected to flow down by merging, and up by copying. This “merge down, copy up” practice assures that merging is done only when necessary, and always in the more forgiving of the two streams involved.
Merging means incorporating another stream’s changes into your stream, and can require you to resolve conflicts. Copy propagates a duplicate of the source stream to the target. The following diagram shows a basic stream hierarchy: changes are merged down (to streams of lesser stability) and copied up (to streams of greater stability):
The following table summarizes these qualities of stream types:
| Stream Type | Stability | Merge | Copy |
|---|---|---|---|
|
|
Stable per your policy (for example, all code builds) |
from child (from release, or to development) |
to child (to release, or from development) |
|
|
N/A; used to filter streams |
N/A |
N/A |
|
|
Unstable |
from parent |
to parent |
|
|
Unstable |
from parent |
to parent |
|
|
Highly stable |
to parent |
from parent |
Task streams
Task streams are lightweight short-lived streams used for bug fixing or new features that only modify a small subset of the stream data. Since branched (copied) files are tracked in a set of shadow tables that are later removed, repository metadata is kept to a minimum when using this type of stream and server performance is optimized.
They are branches that work just like development streams, but task streams remain semi-private until branched back to the parent stream. Designed as lightweight branches, they are most effective when anticipated work in the branch will only affect a small number of files relative to the number of files in the branch.
Task streams are intended to be deleted or unloaded after use. Because you cannot re-use task stream names even after the stream has been deleted, most sites adopt a naming convention that is likely to be unique for each task, such as user-date-jobnumber.
Working within task streams is just like working in a development stream:
-
Create the task stream (in this example, as a child of a development stream).
$ p4 stream -t task -P //projectX/dev //Tasks/mybug123 -
Populate the stream.
$ p4 populate -d "Fix bug 123" -S //Tasks/mybug123 -r - Make changes to files in the stream and submit the changes.
-
Merge down any required changes from the parent stream, resolving as necessary.
$ p4 merge -
Copy up the changes you made into the parent stream.
$ p4 copy --from //Tasks/mybug123 -
Delete or unload the task stream.
$ p4 stream -d //Tasks/mybug123Alternatively, use
p4 unloadto unload it:$ p4 unload -s //Tasks/mybug123Use unload if you think you might to work on the task stream again.
Only workspaces associated with the task stream can see all the files in the stream; the stream appears as a sparse branch to other workspaces, which see only those files and revisions that you changed within the task stream. Most other metadata for the task stream remains private.
Task streams can quickly accumulate in a depot until they are deleted or unloaded; to keep a project depot uncluttered by task streams, your Helix administrator or project lead may choose to establish certain streams depots as dedicated holding areas for task streams. In this case, create your stream in the task streams depot as a child of a parent in the project depot.
Task streams are unique in that they can live in different depots from their children or parents. However, the best practice is to have them reside in the same depot as their children or parents.
Virtual streams
Virtual streams can be used to create alternative views of real streams. Virtual streams differ from other stream types in that a virtual stream is not a separate set of files, but instead a filtered view of its parent stream. A virtual stream can have child streams, and its child streams inherit its views.
Stream paths
Stream paths control the files and paths that compose a stream and define how those files are propagated. Except for the mainline, each stream inherits its structure from its parent stream. To modify the structure of the child, you specify the paths as follows:
| Type | Sync? | Submit? | Integrate to/from Parent? | Remarks |
|---|---|---|---|---|
|
|
Y |
Y |
Y |
(Default) For files that are edited and propagated between parent and child streams. All files in a shared path are branched and, in general, shared paths are the least restricted. |
|
|
Y |
Y |
N |
For files that must not be propagated outside the stream but can be edited within it, such as binary build results. |
|
|
Y |
N |
N |
For files that must be physically present in the stream but are never changed.
Example: third-party libraries. Import paths can reference a specific
changelist (or a label that aliases a changelist) to limit the imported files
to the revisions at that change or lower. Use the syntax @changelist#, as
in: |
|
|
Y |
Y |
N |
Functions like an import path, in that it can reference an explicitly-defined
depot path, but unlike a standard import path, you can submit changes to the
files in an |
|
|
N |
N |
N |
Files in the parent stream that must never be part of the child stream. |
In the following example, files in the src path are not submittable (and are
imported from the parent stream’s view), files in the lib path are not
submittable (and are imported from an explicitly-specified location in the
depot), and files in the db path can be edited and submitted in the stream,
but can never be copied to the parent:
Paths:
share ...
import src/...
import lib/... //depot/lib3.0/...
isolate db/...The paths are used to generate the mappings for workspaces that are associated
with the stream. If the stream structure changes, the workspace views are
updated automatically and in fact cannot be altered manually. If the stream is
locked, only the stream owner (or stream owners, if the Owner: field of the
stream is set to a group) can edit the stream specification.
Stream specification can also remap file locations (so that a file in specified depot location is synced to a different location in the workspace) and screen out files according to file type. For example, to ensure that object files and executables are not part of the stream, add the following entries to the stream specification:
Ignored:
.o
.exeStream paths and inheritance between parents and children
Child streams inherit folder paths and behavioral rules from their parents. When we talk about inheritance between parents and children, it helps to think in the following terms:
-
Permissiveness: what actions (submit, sync, etcetera) are permitted on a path?
Path types are inherited from parent streams, and you cannot override the effects of the path types assigned by parent streams. In other words, child streams are always as permissive or less permissive than their parents, but never more permissive. For example, if a parent stream defines a path as
isolate, its child streams cannot redefine the path asshareto enable integrations. -
Inclusiveness: what paths are included in the stream?
Since children cannot, by definition, be more inclusive than their parents, you cannot include a folder path in a child that is not also included in its parent. This means, for example, that you cannot add an
isolatepath to a child if the folders in that path are not also included in the parent.In the example in the table below, the incorrectly defined
Devstream, which is a child ofMain, contains anisolatepath that does not work, because it includes folders that are not included in the parent. In order to isolate theconfig/folder in theDevstream, that folder has to be included as ashareorisolatepath inMain:Incorrect Correct Stream: //Acme/Main Parent: none Paths: share apps/... Paths: share tests/... Stream: //Acme/Dev Parent: //Acme/Main Paths: share apps/... share tests/... isolate config/...Stream: //Acme/Main Parent: none Paths: share apps/... share tests/... share config/... Stream: //Acme/Dev Parent: //Acme/Main Paths: share apps/... share tests/... isolate config/...
Example 22. Simple share
Let’s start with a simple case: two streams, //Ace/main and its child
//Ace/dev.
Stream: //Ace/main Parent: none Paths: share ... Stream: //Ace/dev Parent: //Ace/main Paths: share ...
In this case, the entire stream path is shared. When you switch your workspace
to the //Ace/main stream, the workspace view looks like this:
//Ace/main/... //bruno_ws/...
The workspace view maps the root of the //Ace/main stream to your workspace.
When you you switch your workspace to the //Ace/dev stream, the workspace view
is this:
//Ace/dev/... //bruno_ws/...
And the branch view for //Ace/dev/ looks like this:
//Ace/dev/... //Ace/main/...
In other words, the entire dev stream can be synced to workspaces, and the
entire stream can be branched, merged, and copied.
Example 23. Share and import
Let’s look at an example where software components are housed in three separate
depots: //Acme, //Red, and //Tango.
The Acme mainline is configured like this:
Stream: //Acme/Main
Parent: none
Paths: share apps/...
share tests/...
import stuff/... //Red/R6.1/stuff/...
import tools/... //Tango/tools/...If you switch your workspace to the //Acme/Main stream, this would be your
workspace view:
//Acme/Main/apps/... //bruno_ws/apps/... //Acme/Main/tests/... //bruno_ws/tests/... //Red/R6.1/stuff/... //bruno_ws/stuff/... //Tango/tools/... //bruno_ws/tools/...
The stream’s Paths field lists folders relative to the root of the stream.
Those are the folders you get in your workspace, beneath your workspace root.
The shared folders are mapped to the //Acme/Main path, and the imported paths
are mapped to their locations in the //Red and //Tango depots.
Example 24. Share, isolate, exclude, and import
Let’s say that your team doesn’t want to do actual development in the mainline. In this example, XProd feature team has a development stream of their own, defined like this:
Stream: //Acme/XProd
Parent: //Acme/Main
Paths: import ...
isolate apps/bin/...
share apps/xp/...
exclude tests/...Switching your workspace to the //Acme/XProd stream gives you this view:
//Acme/Main/apps/... //bruno_ws/apps/... //Acme/XProd/apps/bin/... //bruno_ws/apps/bin/... //Acme/XProd/apps/xp/... //bruno_ws/apps/xp/... //Red/R6.1/stuff/... //bruno_ws/stuff/... //Tango/tools/... //bruno_ws/tools/... -//Acme/XProd/tests/... //bruno_ws/tests/...
Here we see workspace view inheritance at work. The contents of imported paths are mapped into your workspace. The shared and isolated paths are mapped to the child stream; these contain the files the XProd team is working on and will be submitting changes to. And the excluded path (marked with a minus sign in the view) doesn’t appear in the workspace at all.
Because the //Acme/XProd stream has a parent, it has a branch mapping that can
be used by the copy and merge commands. That branch view consists of the
following, with just one path shared by the child and parent.
-//Acme/XProd/apps/... //Acme/Main/apps/... -//Acme/XProd/apps/bin/... //Acme/Main/apps/bin/... //Acme/XProd/apps/xp/... //Acme/Main/apps/xp/... -//Acme/XProd/stuff/... //Acme/Main/stuff/... -//Acme/XProd/tests/... //Acme/Main/tests/... -//Acme/XProd/tools/... //Acme/Main/tools/...
When you work in an //Acme/XProd workspace, it feels as if you’re working in a
full branch of //Acme/Main, but the actual branch is quite small.
Example 25. Child that shares all of the above parent
Let’s suppose that Lisa, for example, creates a child stream from
//Acme/XProd. Her stream spec looks like this:
Stream: //Acme/LisaDev Parent: //Acme/XProd Paths: share ...
Lisa’s stream has the default view template. Given that Lisa’s entire stream path
is set to share, you might expect that her entire workspace will be mapped to
her stream. But it is not, because inherited behaviors always take precedence;
sharing applies only to paths that are shared in the parent as well. A workspace
for Lisa’s stream, with its default view template, has this client view:
//Acme/Main/apps/... //bruno_ws/apps/... -//Acme/LisaDev/tests/... //bruno_ws/tests/... //Acme/LisaDev/apps/bin/... //bruno_ws/apps/bin/... //Acme/LisaDev/apps/xp/... //bruno_ws/apps/xp/... //Red/R6.1/stuff/... //bruno_ws/stuff/... //Tango/tools/... //bruno_ws/tools/...
A workspace in Lisa’s stream is the same as a workspace in the XProd stream,
with one exception: the paths available for submit are rooted in
//Acme/LisaDev. This makes sense; if you work in Lisa’s stream, you expect to
submit changes to her stream. By contrast, the branch view that maps the
//Acme/Dev stream to its parent maps only the path that is designated as
shared in both streams:
-//Acme/Main/apps/... //XProd/apps/... -//Acme/LisaDev/tests/... //XProd/tests/... -//Acme/LisaDev/apps/bin/... //XProd/apps/bin/... //Acme/LisaDev/apps/xp/... //bruno_ws/apps/xp/... -//Red/R6.1/stuff/... //XProd/stuff/... -//Tango/tools/... //XProd/tools/...
The default template allows Lisa to branch her own versions of the paths her team is working on, and have a workspace with the identical view of non-branched files that she would have in the parent stream.
Update streams
As part of maintaining your version control application, you will likely update streams over time, by changing any of the fields listed above, to do such things as:
-
modify the paths the stream consumes when the stream proves to be too narrow or too wide, in order to:
- change the version of an included library by modifying the target of an
importpath - change the scope of a path to widen or narrow the scope included
- change the version of an included library by modifying the target of an
- Change restrictions on who can submit to the stream
To do this, you modify stream specifications directly via the p4 stream
command, automatically and immediately updating all workspace views derived from
that stream.
Make changes to a stream spec and associated files atomically
Alternatively, you can isolate edits to the stream spec to the editing client prior to making them available to other clients as part of an atomic changelist submission. This works just as edits to files do: they are made locally on a single client and then submitted to make them available to other clients.
This functionality has a couple of important benefits:
- You can stage a stream spec in your workspace and test it before submitting it.
- You can submit the spec atomically in a changelist along with a set of files. Since the stream structure dictates the workspace view, this means that when users sync, they obtain the new view and the new files together.
You open and submit changes to the stream spec using the following three commands:
p4 stream editputs the client’s current stream spec into theopenedstate, isolating any edits made to fields that affect view generation. While the spec is open, those fields are marked with the comment#opento indicate that they are open and isolated to your client. Changes made to these fields affect your workspace view normally, but other clients are not affected.p4 stream resolveresolves changes that have been submitted to the stream spec by other users since you opened it. You may not submit changes to the stream spec until newer changes have been resolved.p4 stream revertreverts any pending changes made to the open spec, returning your client to the latest submitted version of the stream.
For details on all three of these commands, see the p4 stream page in the
P4 Command Reference.
By default, the open stream spec is included along with files that are
shelved or submitted in a changelist. Conversely, when unshelving a change that
contains an open stream spec, the current stream is opened and the shelved
version becomes the opened version. If the stream is already open when
attempting to unshelve, a warning is generated and the unshelve operation
aborts. The stream may be omitted from any of these operations by using the -Af
flag to specify that only files should be acted upon.
See the p4 submit, p4 shelve, and p4 unshelve commands in the
P4 Command Reference for details.