File References

About 1350 wordsAbout 5 min

Background

Pipeline configuration files can reference other files for pipeline templates, environment variables, task parameters, etc.

Referenced files may contain sensitive information, so permission checks are needed to prevent information leakage.

Reference Methods

CNB supports four types of configuration file references:

Pipeline template reference: include
Environment variable reference: imports
Built-in task parameter reference: optionsFrom
Plugin task parameter reference: settingsFrom

Note:

File paths declared in imports, optionsFrom and settingsFrom support environment variables. You can declare environment variables in the pipeline first, then reference them in tasks.

.cnb.yml
main:
  push:
    - env:
        CNB_CONFIG_URL: https://cnb.cool/<your-repo-slug>/-/blob/main/xxx
        CNB_CONFIG_FILE: account.yml
      imports:
        # Assume env1.yml declares variable CNB_ENV_FILE_URL
        - https://cnb.cool/<your-repo-slug>/-/blob/main/xxx/env1.yml
        # File paths after imports can use variables declared in previous files
        - ${CNB_ENV_FILE_URL}/env2.yml
      stages:
        - name: echo
          script: echo 1
        - name: Built-in task
          type: some-type
          optionsFrom:
            - ${CNB_CONFIG_URL}/${CNB_CONFIG_FILE}
        - name: Plugin task
          image: some-image
          settingsFrom:
            - ${CNB_CONFIG_URL}/${CNB_CONFIG_FILE}

Additionally, optionsFrom and settingsFrom support reading local files. Even if ./path/to/file doesn't exist in the repository, you can write it in the pipeline for loading during execution, like:

.cnb.yml
main:
  push:
    - env:
        CNB_CONFIG_URL: https://xxx.com/p1/xxx
        CNB_CONFIG_FILE: account.yml
      stages:
        - name: echo
          script: echo some-content > ./path/to/file-not-in-git
        - name: Built-in task
          type: some-type
          optionsFrom:
            # Reference file generated by pipeline but not in git
            - ./path/to/file-not-in-git
        - name: Plugin task
          image: some-image
          settingsFrom:
            # Reference file existing in git
            - ./path/to/file-in-git

Permission Checks

For configuration files in public or same-origin repositories, pipelines can reference them directly.

For private and non-origin repositories, you can declare four fields: allow_slugs, allow_events, allow_branches, and allow_images to control the access scope. All four fields support glob pattern strings (multiple expressions separated by , or |) or string arrays.

Glob Matching Rules

* matches any character except /. For example: my-group/* matches my-group/repo but not my-group/sub/repo.
** matches any character including /. For example: my-group/** matches both my-group/repo and my-group/sub/repo.

Use ** when you need to match paths containing / (e.g., repositories under nested groups).

For example:

# String format, multiple expressions separated by ',' or `;`
allow_slugs: "group/project,some/another-project/feature-*"
# String array
allow_events:
  - push
  - tag_push
  - "tag_deploy.*"
# Only allow specified image.
allow_images: "cnbcool/ssh"
# Allow branches starting with main or feature/
allow_branches:
  - main
  - "feature/*"

It is recommended to declare multiple expressions using string arrays for better readability. If the configuration file is in a text format that does not support arrays, use the string format.

If the above four fields are not declared, it checks whether the pipeline trigger is a developer or higher in the target repository. If yes, access is granted; otherwise, access is denied.

Where:

allow_slugs: specifies which repositories' pipelines are allowed to read the current file.
allow_events: specifies under which events pipelines can read the current file. Configurable event names refer to Trigger Events.
allow_branches: specifies under which branches pipelines can read the current file.
allow_images: specifies which image plugins are allowed to read the current file.

The permission check flow for whether a pipeline can load target files is shown below:

Flow explanation:

Same source: Pipeline and target file belong to same repository
Allow check: Declared allow fields checked sequentially, all must pass
File read permission: See Role Permissions

Explanation of Allow Requirement Checks:

In untrusted events, the pipeline configuration or trigger is uncontrollable. The configuration file must include allow_events to be referenced by untrusted event pipelines.
Pipelines may use third-party plugins. For security, the configuration file must declare allow_images to be referenced by plugin tasks.

Branch checked in allow_branches is same as CNB_BRANCH.

Image check notes:

Plugin task parameters can be set via settingsFrom or imported as env vars via imports. Both count as plugin task file references.
When the image name has no tag or has the :latest tag, it matches allow_images using both image-name and image-name:latest. Either match is valid.
Non-plugin task image is empty string and cannot match any glob pattern. Files with allow_images cannot be referenced by non-plugin tasks.
Note: Jobs with both image and script are script tasks using image as execution environment
Pipelines can declare image as build environment but not as plugin task. Files with allow_images cannot be referenced at pipeline level.

Examples

Pipeline Secret File Reference

# Configuration file secret.yml in the keystore repository
allow_slugs: "p1/**"
allow_events: push
allow_branches: main
# DockerHub username and password
DOCKER_USER: docker-user
DOCKER_PWD: docker-pwd

.cnb.yml
# Pipeline configuration file .cnb.yml
main:
  push:
    - services:
        - docker
      imports: https://cnb.cool/<your-repo-slug>/-/blob/main/xxx/secret.yml
      stages:
        - name: docker login
          script: |
            # Log in to DockerHub
            docker login -u ${DOCKER_USER} -p "${DOCKER_PWD}"
            # docker build xxx
            # docker push xxx

The pipeline references secret.yml from another repository, but the trigger does not have read access to this file.

secret.yml declares allow_slugs, allow_events, and allow_branches: if a repository under the p1 organization triggers a push event on the main branch, the pipeline can reference this secret file.

This ensures that only the owner or administrator of the secret repository can view or modify the sensitive content, while still allowing it to be referenced by pipelines that meet the specified conditions.

Plugin Task Parameter Reference

# Configuration file image-settings.yml in the keystore repository
allow_images: "registry.com/image1/**"
allow_slugs: "p1/**"
arg1: arg1
arg2: arg2

# Pipeline configuration file .cnb.yml configuring a plugin task
name: image job
image: registry.com/image1/print:latest
settingsFrom:
  - https://cnb.cool/<your-repo-slug>/-/blob/main/xxx/image-settings.yml

If the image is created by a third party, there may be a risk of information leakage. You can use allow_images to restrict the configuration file to plugin tasks with specific image names.

The image registry.com/image1/print:latest matches registry.com/image1/**, so the image check passes. allow_slugs further restricts usage to pipelines under the p1 organization.