File References
About 1314 wordsAbout 4 min
Background
Pipeline configuration files can reference other files for pipeline configuration, environment variables, task parameters, etc.
Since files may contain sensitive information, permission checks are needed for file references to prevent information leakage.
Reference Methods
CNB supports four types of configuration file references:
- Pipeline template reference: include
- Environment variable reference: imports
- Built-in task parameter reference: optionsFrom
- Plugin task parameter reference: settingsFrom
Note:
For flexible pipeline task parameter configuration, file paths declared in imports
, optionsFrom
and settingsFrom
support environment variables.
You can declare environment variables in the pipeline first, then reference them in tasks, like:
main:
push:
- env:
CNB_CONFIG_URL: https://cnb.cool/<your-repo-slug>/-/blob/main/xxx
CNB_CONFIG_FILE: account.yml
imports:
# Assume env1.yml declares variable CNB_ENV_FILE_URL
- https://cnb.cool/<your-repo-slug>/-/blob/main/xxx/env1.yml
# File paths after imports can use variables declared in previous files
- ${CNB_ENV_FILE_URL}/env2.yml
stages:
- name: echo
script: echo 1
- name: Built-in task
type: some-type
optionsFrom:
- ${CNB_CONFIG_URL}/${CNB_CONFIG_FILE}
- name: Plugin task
image: some-image
settingsFrom:
- ${CNB_CONFIG_URL}/${CNB_CONFIG_FILE}
Additionally, optionsFrom
and settingsFrom
support reading local files. Even if ./path/to/file
doesn't exist in the repository, you can write it in the pipeline for loading during execution, like:
main:
push:
- env:
CNB_CONFIG_URL: https://xxx.com/p1/xxx
CNB_CONFIG_FILE: account.yml
stages:
- name: echo
script: echo some-content > ./path/to/file-not-in-git
- name: Built-in task
type: some-type
optionsFrom:
# Reference file generated by pipeline but not in git
- ./path/to/file-not-in-git
- name: Plugin task
image: some-image
settingsFrom:
# Reference file existing in git
- ./path/to/file-in-git
File Types
In CNB, three types of configuration files are supported for reference:
YAML Files
Files with .yml
or .yaml
extension, like:
account.yml
SOME_ACCOUNT: some-account
SOME_PASSWORD: some-password
JSON Files
Files with .json
extension, like:
account.json
{
"SOME_ACCOUNT": "some-account",
"SOME_PASSWORD": "some-password"
}
Text Files
Files not of the above two types are parsed as text files, like:
account.txt
SOME_ACCOUNT=some-account
SOME_PASSWORD=some-password
Permission Checks
For configuration files in public or same-origin repositories, pipelines can reference them directly.
For configuration files in private and non-origin repositories, four fields can be declared: allow_slugs
, allow_events
, allow_branches
, and allow_images
to control accessible scopes. All four fields accept either string
or string array
values in glob
pattern format.
If none of these four fields are declared, the system will check whether the pipeline triggerer has read permission
for the repository owning the configuration file.
Where:
allow_slugs
specifies which repositories' pipelines are allowed to read the current file.allow_events
specifies under which events pipelines can read the current file. Configurable event names refer to Trigger Events.allow_branches
specifies under which branches pipelines can read the current file.allow_images
specifies which image plugins are allowed to read the current file.
The permission check flow for whether a pipeline can load target files is shown below:
Overall flow:
Flow explanation:
- Same source: Pipeline and target file belong to same repository
- Allow check: Declared allow fields checked sequentially, all must pass
- File read permission: Pipeline trigger has read permission for referenced file's repository, see Role Permissions
Allow necessity check explanation:
- "Pull request" events include
pull_request
andpull_request.update
events - In PRs, especially from forked repos, source branch code is untrusted without admin review. For security, config files must have
allow_events
for pull_request pipelines - Pipelines can use third-party plugins. For security, config files must declare
allow_images
for plugin tasks
Branch checked in allow_branches
is same as CNB_BRANCH.
Image check notes:
- Plugin task parameters can be set via
settingsFrom
or imported as env vars viaimports
. Both count as plugin task file references - When image name has no tag, both
name
andname:latest
are matched againstallow_images
. Match either passes - Non-plugin task image is empty string, can't match any glob pattern. Config files with
allow_images
can't be referenced by non-plugin tasks - Note: Jobs with both image and script are script tasks using image as execution environment
- Syntactically pipelines can declare image as build environment but not as plugin task. Config files with
allow_images
can't be referenced at pipeline level
Examples
Pipeline Secret File Reference
# Configuration file secret.yml in the keystore repository
allow_slugs:
- "p1/**"
allow_events:
- push
allow_branches:
- main
# DockerHub username and password
DOCKER_USER: docker-user
DOCKER_PWD: docker-pwd
# Pipeline configuration file .cnb.yml
main:
push:
- services:
- docker
imports: https://cnb.cool/<your-repo-slug>/-/blob/main/xxx/secret.yml
stages:
- name: docker login
script: |
# Log in to DockerHub
docker login -u ${DOCKER_USER} -p "${DOCKER_PWD}"
# docker build xxx
# docker push xxx
The pipeline references the configuration file secret.yml
from another keystore repository, but the pipeline triggerer does not have read access to this file.
The referenced file secret.yml
declares allow_slugs
, allow_events
, and allow_branches
.
If a repository under the p1
organization triggers a push
event on the main
branch, the pipeline can reference this secret file and use the sensitive information within it.
This ensures that only the owner or administrator of the keystore repository can view or modify the sensitive content in secret.yml
, while still allowing it to be referenced by pipelines that meet the specified conditions.
Plugin Task Parameter Reference
# Configuration file image-settings.yml in the keystore repository
allow_images:
- "registry.com/image1/**"
allow_slugs:
- "p1/**"
arg1: arg1
arg2: arg2
# Pipeline configuration file .cnb.yml configuring a plugin task
name: image job
image: registry.com/image1/print:latest
settingsFrom:
- https://cnb.cool/<your-repo-slug>/-/blob/main/xxx/image-settings.yml
If the image is created by a third party, there may be a risk of information leakage. This can be mitigated by using allow_images
to restrict the configuration file to only be used by plugin tasks with specific image names that are deemed safe.
The image registry.com/image1/print:latest
matches registry.com/image1/**
, so the image check passes.
allow_slugs
specifies that the configuration file can only be referenced by pipelines of repositories under the p1
organization, further controlling the scope of the configuration file's usage.