PyPI Artifact Registry

About 1217 wordsAbout 4 min

The PyPI Artifact Registry is used for storing and distributing Python packages. Developers can upload, manage, and install various Python third-party libraries and tools through it.

Prerequisites

Local Development

Configure Pull Credentials

Option 1: Add pip source configuration in ~/.pip/pip.conf (MacOS/Linux) or %HOME%\pip\pip.ini (Windows)

[global]
# YOUR_TOKEN: Access token password; REPO_URL: Artifact Registry address
# Example: index-url = https://cnb:<YOUR_TOKEN>@pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
index-url = https://cnb:<YOUR_TOKEN>@<REPO_URL>

Option 2: You can also use command-line setup

# YOUR_TOKEN: Access token password; REPO_URL: Artifact Registry address
# Example: pip config set global.index-url https://cnb:<YOUR_TOKEN>@pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
pip config set global.index-url https://cnb:<YOUR_TOKEN>@<REPO_URL>

Configure Push Credentials

twine configuration

Add the following configuration in ~/.pypirc file

[distutils]
index-servers =
  cnb-pypi-repo
[cnb-pypi-repo]
# Artifact Registry address
repository: <REPO_URL>
username: cnb
# Access token password
password: <YOUR_TOKEN>

Example:

[distutils]
index-servers =
  cnb-pypi-repo
[cnb-pypi-repo]
repository: https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
username: cnb
password: "<YOUR_TOKEN>"

poetry configuration

Run the following commands

poetry config http-basic.cnb-pypi-repo cnb <YOUR_TOKEN> # Access token password
poetry config repositories.cnb-pypi-repo <REPO_URL> # Artifact Registry address

Example:

poetry config http-basic.cnb-pypi-repo cnb <YOUR_TOKEN>
poetry config repositories.cnb-pypi-repo https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple

Pull Artifacts

#Example 1: pip install requests
#Example 2: Pull a specific version pip install requests==2.30.1
pip install <PACKAGE_NAME>

# Example, pull from a temporary source pip install -i https://cnb:<YOUR_TOKEN>@pypi.cnb.cool/cnb/pypi-repo/-/packages/simple requests
pip install -i https://cnb:<YOUR_TOKEN>@<REPO_URL> <PACKAGE_NAME>

Build and Push Artifacts

twine

python -m build  
twine upload -r cnb-pypi-repo dist/*

poetry

poetry build
poetry publish -r cnb-pypi-repo

Cloud Native Build

Pull Artifacts

Option 1: Use built-in CNB_TOKEN environment variable:

.cnb.yml
main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest 
      stages:
        - name: Pull artifacts
          script:
            # Artifact Registry address
            # Example: pip config set global.index-url https://cnb:${CNB_TOKEN}@pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
            - pip config set global.index-url https://cnb:${CNB_TOKEN}@<REPO_URL>
            # Example: pip install requests
            - pip install <PACKAGE_NAME>

Option 2: Use secret registry file:

.cnb.yml
main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      imports:
        # Import secret repository configuration file
        - <SECRET_REPO_URL> 
      stages:
        - name: Pull artifacts
          script:
            - pip config set global.index-url https://<USER_NAME>:<PASS_WORD>@<REPO_URL>
            - pip install <PACKAGE_NAME>

Example:

.cnb.yml
main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      imports:
        - https://cnb.cool/cnb-demo/env-demo/-/blob/main/envs/env.yml 
      stages:
        - name: Pull artifacts
          script:
            # Import secret registry file and inject USER_NAME and PASS_WORD as environment variables
            - pip config set global.index-url https://${USER_NAME}:${PASS_WORD}@pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
            - pip install requests

Push Artifacts

twine configuration

Option 1: Use built-in CNB_TOKEN environment variable:

.cnb.yml
main: 
  push: 
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      stages:
        - name: Build & Upload to repository
          script:
            - export TWINE_USERNAME=${CNB_TOKEN_USER_NAME} 
            - export TWINE_PASSWORD=${CNB_TOKEN}
            # Artifact Registry address
            # Example: export TWINE_REPOSITORY_URL=https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple 
            - export TWINE_REPOSITORY_URL=<REPO_URL>
            - python -m build
            - twine upload dist/*

Option 2: Use the secret repository file

.cnb.yml
main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      imports:
        # Secret repository file address
        # Example: https://cnb.cool/cnb-demo/env-demo/-/blob/main/envs/env.yml
        - <SECRET_REPO_URL> 
      stages:
        - name: Package & Upload to Registry
          script:
            # Secret repository file USER_NAME configuration
            - export TWINE_USERNAME=${USER_NAME}
            # Secret repository file PASS_WORD configuration
            - export TWINE_PASSWORD=${PASS_WORD}
            # Artifact Registry address
            # Example: export TWINE_REPOSITORY_URL=https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
            - export TWINE_REPOSITORY_URL=<REPO_URL>
            - python -m build
            - twine upload dist/*

poetry configuration

Option 1: Use built-in CNB_TOKEN environment variable:

.cnb.yml
main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest 
      stages:
        - name: Build & Upload to repository
          script:
            - export POETRY_HTTP_BASIC_CNB_PYPI_REPO_USERNAME=${CNB_TOKEN_USER_NAME}
            - export POETRY_HTTP_BASIC_CNB_PYPI_REPO_PASSWORD=${CNB_TOKEN}
            # Artifact Registry address
            # Example: poetry config repositories.cnb-pypi-repo https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
            - poetry config repositories.cnb-pypi-repo <REPO_URL>
            - poetry build
            - poetry publish -r cnb-pypi-repo

Option 2: Use the secret repository file:

.cnb.yml
main:
  push:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      imports:
        # Secret registry file address, example https://pypi.cnb.cool/cnb-demo/env-demo/-/blob/main/envs/env.yml 
        - <secret_repo_url>
      stages:
        - name: Build & Upload to repository
          script:
            # Secret repository file USER_NAME configuration
            - export POETRY_HTTP_BASIC_CNB_PYPI_REPO_USERNAME=${USER_NAME}
            # Secret repository file PASS_WORD configuration
            - export POETRY_HTTP_BASIC_CNB_PYPI_REPO_PASSWORD=${PASS_WORD}
            # Artifact Registry address
            # Example: poetry config repositories.cnb-pypi-repo https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
            - poetry config repositories.cnb-pypi-repo <REPO_URL>
            - poetry build
            - poetry publish -r cnb-pypi-repo

Workspaces

Prerequisites

Add the following configuration in .cnb.yml file

.cnb.yml
$:
  vscode:
    - docker:
        image: docker.cnb.cool/examples/language/python-3:latest
      services:
        - vscode
        - docker
      # If using the secret repository file, fill in the following configuration  
      imports:
        # Example: https://cnb.cool/cnb-demo/env-demo/-/blob/main/envs/env.yml
        - <SECRET_REPO_URL>

Configure Pull Credentials

Option 1: Use built-in CNB_TOKEN environment variable:

# Example: pip config set global.index-url https://cnb:${CNB_TOKEN}@pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
pip config set global.index-url https://cnb:${CNB_TOKEN}@<REPO_URL>

Option 2: Use the secret repository file:

# Example: pip config set global.index-url https://${USER_NAME}:${PASS_WORD}@pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
pip config set global.index-url https://${USER_NAME}:${PASS_WORD}@<REPO_URL>

Configure Push Credentials

Enter the following commands in the Workspaces TERMINAL

twine

Option 1: Use built-in CNB_TOKEN environment variable:

export TWINE_USERNAME=${CNB_TOKEN_USER_NAME} 
export TWINE_PASSWORD="${CNB_TOKEN}"
# Artifact Registry address
# Example: export TWINE_REPOSITORY_URL=https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple 
export TWINE_REPOSITORY_URL=<REPO_URL>

Option 2: Use the secret repository file:

# Secret repository file USER_NAME configuration
export TWINE_USERNAME="${USER_NAME}" 
# Secret repository file PASS_WORD configuration
export TWINE_PASSWORD="${PASS_WORD}"
# Artifact Registry address
# Example: export TWINE_REPOSITORY_URL=https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple   
export TWINE_REPOSITORY_URL=<REPO_URL>

poetry

Option 1: Use the built-in CNB_TOKEN environment variable:

export POETRY_HTTP_BASIC_CNB_PYPI_REPO_USERNAME=${CNB_TOKEN_USER_NAME} 
export POETRY_HTTP_BASIC_CNB_PYPI_REPO_PASSWORD="${CNB_TOKEN}"
# Artifact Registry address
# Example: poetry config repositories.cnb-pypi-repo https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple 
poetry config repositories.cnb-pypi-repo <REPO_URL>

Option 2: Use the secret repository file:

# Secret repository file USER_NAME configuration
export POETRY_HTTP_BASIC_CNB_PYPI_REPO_USERNAME="${USER_NAME}"
# Secret repository file PASS_WORD configuration
export POETRY_HTTP_BASIC_CNB_PYPI_REPO_PASSWORD="${PASS_WORD}"
# Artifact Registry address 
# Example poetry config repositories.cnb-pypi-repo https://pypi.cnb.cool/cnb/pypi-repo/-/packages/simple
poetry config repositories.cnb-pypi-repo <REPO_URL>

Pull Artifacts

#Example 1: pip install requests
#Example 2: Pull a specific version pip install requests==2.30.1
pip install <PACKAGE_NAME>

# Example: Pull from a temporary source pip install -i https://cnb:${CNB_TOKEN}@pypi.cnb.cool/cnb/pypi-repo/-/packages/simple requests
pip install -i https://cnb:${CNB_TOKEN}@<REPO_URL> <PACKAGE_NAME>

Build and Push Artifacts

twine

python -m build  
twine upload -r cnb-pypi-repo dist/*

poetry

poetry build
poetry publish -r cnb-pypi-repo

FAQ

Q: Why do I get a 409 error when uploading?

A: The package name and version already exist. Set the "Version Overwrite Strategy" to "Allow overwriting of existing versions", or change the package version and re-upload.

Q: Why do I get a 400 error when uploading?

A: The artifact filename does not comply with PyPI specifications. Please check and fix it before re-uploading.

Q: What PyPI artifact formats does CNB support?

A: Only wheel and sdist formats are supported. Older formats such as egg or exe will be rejected.

More Usage

For more PyPI usage, please refer to the PyPI official documentation.