Helix Core is now P4. Product user interfaces will be updated soon. Learn more >>

P4 CLI Documentation | 2025.1

# Sparse streams

Sparse streams have significant advantages over Task streams.

| Sparse Streams | Task Streams
--- | --- | ---
Performance | Lightweight: no need to wait a long time for the initial populate command, and no need to delete or unload to maintain performance. | Only lightweight after integrating changes from a task stream, and then deleting or unloading the task stream. Otherwise database metadata Information that P4 Server maintains, such as who created file revisions in the depot, whether the file is a 'lazy copy,, the current state of client workspaces, protections, groups, users, labels, streams, and branches. Metadata is stored in the server database and is separate from the 'archive files' that users submit from their client workspace into the depot. accumulation can affect performance.
Flexibility | `sparsedev` can be converted to `development` `sparserel` can be converted to `release` See the `p4 stream` command for `convertsparse`. | `task` can be converted to `development`
Control | Supports setting a configurable limit on the number of branched (noun) A set of related files that exist at a specific location in the P4 depot as a result of being copied to that location, as opposed to being added to that location. A group of related files is often referred to as a codeline. To associate code reviews in P4 Code Review (formerly Helix Swarm) with the projects they are part of, add the 'branch' paths in the P4 Code Reviewproject. (verb) To create a codeline by copying another codeline with the 'p4 integrate', 'p4 copy', or 'p4 populate' command. files to preserve good performance. See `dm.stream.sparse.branchmax` in the P4 CLI Reference. | Retroactive control by deleting or unloading task streams to reduce metadata load.
Code Review | Works with P4 Code Review | Does not work with P4 Code Review
Virtual Stream | Can be the parent of a virtual stream. | Does not support virtual streams.
Private editing | Supports both private and public editing of the stream spec. Use cases include setting the ChangeView limit downwards, adding import paths (or changing paths in general), adding custom fields, and changing the description. To learn more, see Private editing of streams in the P4 CLI Reference. | Does not support private editing of stream spec.

### Reduction of metadata load

Files in sparse streams are only branched as needed, so less metadata needs to be maintained, stored, and sent to downstream servers.

### Productivity

Sparse streams allow you to work with a new stream quickly.

If your organization has a "branch per bug" workflow, sparse streams might accelerate productivity.

Suppose you only need to edit one file in a parent stream with one million files. Instead of waiting several hours for the `development` stream to be created and populated, a sparse stream is ready in less than a minute.

The benefit of being able to start your work quickly might be particularly dramatic in a multi-server environment. If your client workspace is on a remote edge server, network latency to populate a local `development` stream from the commit serve increases the amount of time it takes for your local client workspace to be ready for you to begin your work.

### Lightweight

Sparse streams are lightweight because:

- Branching only occurs when a file needs to be changed.

- A client workspace of a sparse stream has view mappings for both the parent and child stream depot paths. Most files that are visible in a sparse stream reside solely in the depot path of the parent stream.

- When the sparse stream is created, the stream is not populated, parent files within a share path are not branched, and the sparse stream contains no depot files. Instead, when you are working in a sparse stream and check out a file from the parent stream, that file is branched into the sparse stream, sync'd locally, and opened for edit.

### Specify files using client or local file paths, not depot paths

The advantage of using sparse streams is that they contain only the files you want to edit, and those files are local to your client workspace. Avoid using depot paths because the depot view is not a direct reflection of the contents of the sparse stream.

### Do not run p4 populate on a newly-created sparse stream

A sparse stream should only branch a small number of files from its parent stream. The p4 populate command would fully populate the sparse stream and thus prevent the sparse stream from being sparse.

## sparsedev and sparserel

A sparse stream can be a child of a `mainline`, `development`, or `release` stream.

The two types of sparse streams are `sparsedev` and `sparserel`.

In terms of Flow rules :

- `sparsedev` is similar to `development`

- `sparserel` is similar to `release`

Similar but with differences: see Integrating to and from sparse streams.

You can convert a `sparsedev` stream to `development`, and a `sparserel` stream to `release`. See Convert a sparse stream to a non-sparse stream.

You can use either type by itself or in combination with the other type.

## Workflow example

This scenario describes a workflow that combines the use of `sparserel` and `sparsedev`.

In this scenario: 1. A `sparsedev` stream avoids a lengthy populate operation from the `mainline` stream, and is well suited for developing a feature involving a relatively small number of files. 2. When the small feature is ready, you can copy up to the `mainline` stream. 3. When the `mainline` stream is complete and stable, you can create a `release` stream to publish a new product version. 4. Later, if that new version of the product needs a patch or hot fix, you might use a `sparserel` stream to avoid a lengthy populate operation from the `release` stream. A workspace client connected to the `sparserel` stream could make the hotfix build, then merge its changes down to the `release` stream. Any `import` of the `release` stream depot path would then pick up the fix. (See `import` at Stream views and paths.) Do not use a sparse stream depot path as an `import` path in another stream because only a small number of files are resident in the sparse stream depot path and the files of the parent stream depot path are missing. A stream component constructs a complete stream view, but sparse streams cannot be Stream Components. |

## Overlay mapping and ChangeView pinning

Sparse streams streamline your work with the combination of:

- An overlay mapping view into the parent stream to avoid a lengthy populate operation

  and

- A ChangeView number acts as limit that isolates your sparse stream work from changes in the parent stream until you merge, copy, or integrate.

Together, the overlay mapping and the limiting ChangeView allow a sparse stream to remain lightweight as edits proceed and changelist number An integer that identifies a changelist. Submitted changelist numbers are ordinal (increasing), but not necessarily consecutive. For example: 103, 105, 108, 109. A pending changelist number might be assigned a different value upon submission.s increment, provided the edits are on a small number of files.

### An overlay mapping view into the parent stream

Suppose a `sparsedev1` stream named `//tools/sparsedev` is the child of a mainstream stream named `//tools/main`. In the `//tools/sparsedev1` stream spec, the `View` field contains

    //tools/main/... ...
    +//tools/sparsedev1/... ...

where `+` indicates an overlay mapping. To learn more, see Map different depot locations to the same workspace location.

### A ChangeView number acts as limit

The `ChangeView` field of the `//tools/sparsedev1` stream spec might look like:

    //tools/main/...@3

where `@3` changeview number specifies the limiting change. This limit isolates your work in the sparse stream from any changes in the parent stream.

The changeview number is either the change number immediately before the sparse stream was created, or the last change prior to a merge, copy, or integrate into the sparse stream.

This limiting change number automatically appears in the `Paths` field. For instance:

    share ...@3

If you add new share paths, the limiting number is appended to them.

The ChangeView limit can only be changed by integrating.

## Integrating to and from sparse streams

A `sparsedev` stream should follow the same flow of change as a `development` stream.

A `sparserel` stream should follow the same flow of change as a `release` stream.

The main difference is that when integrating to a sparse stream, the sparse stream spec is checked out, the spec changeview number is updated, and the sparse stream spec should be submitted.

After syncing the sparse stream, any new versions of the files that were changed on the parent are now visible in the sparse stream.

If the same file is changed in the parent and in the sparse stream, this file is branched to the sparse stream and scheduled for a resolve. In this case, resolve the files and submit the stream spec and the resolved files.

Specify files using client or local file paths, not depot paths.

## Overlays and precedence

The section on Map different depot locations to the same workspace location explains that the overlay mapping takes precedence over that same filename in a preceding view mapping. With sparse streams, the sparse stream view is overlayed on top of the parent view. Therefore, if a file exists both in the overlay mapping of the sparse stream and the preceding mapping of parent stream, when a client workspace does a sync, it gets the sparse stream version, not the parent stream version.

In this example, `//depot/parent` is a development stream that contains both `a.txt` and `b.txt` and `//depot/child` is a sparsedev stream that contains only `a.txt`.

Sparse stream overlay mappings | Non-sparse stream non-overlay mappings
--- | ---
With an overlay mapping, precedence is given to the lower lines, but files in the preceding view lines are not obscured. With the mapping
    //depot/parent/... ...
    +//depot/child/... ...
a sync gets both `//depot/child/a.txt` and `//depot/parent/b.txt` and both files are mapped to the workspace. | With non-overlay mappings, a view line that takes precedence over preceding lines obscures the preceding lines. With the mapping
    //depot/parent/... ...
    //depot/child/... ...
a sync gets only `//depot/child/a.txt` because `//depot/parent/b.txt` is inaccessible.
In the case of `//depot/child/a.txt` and `//depot/parent/a.txt`, the sparse stream overlay view takes precedence, so the workspace is mapped to the sparse stream version. The `//depot/parent/b.txt` file is not obscured.

## Manage growth

Sparse streams are well suited for use cases in which you only need to make changes to a relatively small number of files, such as to develop a small bug fix or to release a small bug fix in a patch or hotfix.

As the number of branched files in a sparse stream increases, the risk of a performance degradation also increases.

#### Limit with configurable

To mitigate that risk, the administrator can control the number of branched files with the `dm.stream.sparse.branchmax` configurable. If an operation would exceed that limit, the operation is blocked and an error message appears. An administrator with the `super` access level can change the limit at any time. If the value of the configurable is reset to a value that is less than the number of branched files in any existing sparse stream, that excessively-branched sparse stream will continue to exist, but its files cannot be edited.

#### Convert to a non-sparse stream

If the number of files that need changes exceed the limit imposed by the `dm.stream.sparse.branchmax` configurable, you can Convert a sparse stream to a non-sparse stream.

Editing a parent file branches the file into the sparse stream. Therefore, even if the first edit of a parent file is subsequently reverted, the file remains branched into the sparse stream. You can, however, remove any branched files that have no changes from the sparse stream by using the `p4 prune` command.

## Unshelve to a sparse stream

To unshelve to a sparse stream from another stream:

1. Ensure that the ChangeView field for the parent stream is up-to-date.

   To see whether the stream spec needs to be updated, use the p4 istat command.

   If the stream spec needs to be updated, do a merge down for a sparsedev stream or a copy up for a sparserel stream.

   This updates the ChangeView field, which, as mentioned in A ChangeView number acts as limit, can only be updated through integration.

2. Follow the usage pattern of Unshelve a shelf owned by another stream in the P4 CLI Reference.

## Restrictions

- A sparse stream cannot be reparented.

- Sparse streams can consume Stream Components, and inherit stream components from a parent, but cannot be consumed as components of other streams.

- If a file has become resident in sparse stream, it remains resident, regardless of whether you submit changes to the file or revert the file.

- Non-sparse streams cannot be converted to sparse streams, even if the stream was originally `sparsedev` or `sparserel` prior to a conversion to `development` or `release`.

- Unsupported commands and features:

  - `p4 unsubmit`, `p4 resubmit`, and `p4 undo` are not supported.

  - Like `task` streams, `sparse` streams are not supported for use with DVCS or graph depots A depot of type graph that is used to store Git repos managed by P4 Server. See also Git Connector and classic depot..

- Integrations:

  - Each integration must be full, that is, with a file spec of `...` because "cherry-picking" a subset of files is not supported.

  - The only revision limit allowed is `@<=change` because the parent depot paths are pinned to a specific change number. The changeview number is automatically updated during integrations into the sparse child. No editing of the changeview number is allowed after any files are populated or added into the sparse stream. The changeview number can be updated only through integration.

- Paths:

  - Paths of type `import` and `import+` observe the change specified in the `import` and `import+` paths rather than the changeview number in a sparse stream. To make the import path’s files match the sparse stream changeview number, use a change specifier such as `import x/... //depot/x/...@N`

  - Each sparse stream must have at least one share path and ParentViews are always `inherit`.

- Virtual streams:

  - The only type of stream that can be a child of a sparse stream is a `virtual` stream.

  - A `virtual` child of a sparse stream cannot have children and cannot be the target of an integration.

  - A sparse stream cannot be a child of a `virtual` stream.

## Convert a sparse stream to a non-sparse stream

If you find that your stream needs more files than allowed (see Manage growth), you can convert:

- From a `sparsedev` stream to a `development` stream.

- From a `sparserel` stream to a `release` stream.

The steps are:

1. `p4 stream convertsparse`

2. `p4 submit -d "converted to a non-sparse type"`

The stream name remains the same, but its type has changed. When the second step, `p4 submit`, has fully completed, the converted stream will be fully branched to contain the metadata for all the files from the parent stream, and the client spec contains no `ChangeView` and no overlay mappings.

Your search for returned results