Skip to content

Publishing

Packaging Goals

G1 - Minimal Installation Effort

Application developers shall access the package with minimal effort from PyPI, e.g.

uv init                # create virtual environment
uv add mtppy2.0        # get package

G2 - Minimal Tool Usage Effort

Package users shall utilize the command line tools with minimal effort, e.g.

mtp_inspector --help                
rmtp_pea --help

G3 - Branches in sync

With the streamlined workflow a clear and failsafe approach is achieved that keeps changes in sync across branches

G4 - Opportunity to publish Release Candidate Packages

During development, there shall be the option to publish a release candidate as package.

Overview Release Flows

The following overview provides a general introduction to the release options:

flowchart TD
    RI[Release Intention]
    ProdR[Productive Release]
    PreR[Pre Release]
    HotR[Hotfix Release]
    PP[PyPI]
    TPP[Test.PyPI]

    %% Section A: Pre-Release Candidate
    subgraph A[Optional: Manual Pre-Release Candidate <br> just with Test.PyPI Account]
        direction TB
        A1[Build package]
        A2[Publish to Test.PyPI with Tag]
        A1 --> A2
    end

    %% Section B: Pre-Release Candidate
    subgraph B[Optional: Automatic Pre-Release Candidate]
        direction TB
        B1[Tag with vX.Y.ZrcN]
        B2[CI/CD detects Tag]
        B3[CI/CD publishes to Test.PyPI]
        B1 --> B2 --> B3
    end

    %% Section C: Production Release
    subgraph C[Production Release]
        direction TB
        C0[Update develop for release]
        C1[Place Merge Request develop into main]
        C2[CI/CD detects merge request]
        C3[CI/CD publishes to Test.PyPI]
        C4[Complete Merge Request]
        C5[Tag main with vX.Y.Z]
        C6[CI/CD detects Tag on main]
        C7[CI/CD publishes to PyPI]
        C8[Merge `main` into `develop`]
        C0 --> C1 --> C2 --> C3 --> C4 --> C5 --> C6 --> C7
        C5 --> C8
    end

    %% Section D: Hotfix
    subgraph D[Hotfix]
        direction TB
        D1[Branch from main, e.g. hotfix/urgent-bug]
        D2[Apply fix and Place Merge Request to main]
        D3[Pass CI/CD & Code Review]
        D4[Complete Merge Request]
        D5[Tag main with patched version]
        D6[CI/CD detects tag]
        D7[CI/CD publishes to PyPI]
        D8[Merge `main` into `develop`]
        D1 --> D2 --> D3 --> D4 --> D5 --> D6 --> D7
        D5 --> D8
    end

    %% Information flows
    RI -.-> PreR
    PreR -.-> B1
    ProdR -.-> C0
    RI -.-> ProdR
    PreR -.-> A1
    RI -.-> HotR
    HotR -.-> D1

    A2 -.-> TPP
    B3 -.-> TPP
    C3 -.-> TPP
    C7 --> PP
    D7 -.-> PP

Workflow for Pre-Releases

According to goal G4, pre-releases of a package during development shall be possible.

Publish to Test.PyPI during development

This procedure works on every branch, building and publishing to Test.PyPI It relies on Tag of branch and automatic CI/CD Pipeline.

  1. Create or switch to your branch
    git checkout `develop`
git checkout -b feature/x.y.z

  2. Changed Version

    • Introduce your changes
    • Check version for a pre-release candidate with pattern vX.Y.Zrc<N> e.g. v1.2.3rc1, where
      • vX.Y.Z is the same as in pyproject.toml to keep track of develop as dev basis
      • N is the incremental index of release candidates
      • rc is ths static flag to identify as release candidate
      • Note: To avoid tag conflicts visit the release history
    • Bump the version in pyproject.tml to the pre-release candidate version
      • Note: Please remove on merge back into develop
    • Test the build process of your package locally 
      uv sync --all-groups
uv build
      Attention: Before execution, dist/* shall be empty to avoid mixing artifacts between builds.

  3. Trigger the publishing of pre-release
    CI/CD will look for release Candidate identified through a Tag of pattern vX.Y.Zrc<N> e.g. v1.2.3rc1 Tag the branch and push your branch to repo including Tag.

git tag -a `vX.Y.Zrc<N>` -m "PreRelease version X.Y.Z candidate N"
git push origin  --tags
  1. Check package built
    The pipeline will detect the Tag and if pipeline is successful publish to Test.PyPI.
    Checkout release history in case your version is not the latest one.
---
title: Release Candidate Flow
---
gitGraph

    checkout main
    commit id: "Last Release" tag: "vX.Y.Z" 
    branch develop
    checkout develop
    commit id: "Some commit of develop"
    branch feature/se
    checkout feature/se
    commit id: "Feature Changes"
    commit id: "Release Candidate Tagging" tag: "vX.Y.Zrc1"

How to use packages from Test.PyPI

If someone intends to use a Test.PyPI package, you can do so via:

uv add --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --prerelease if-necessary-or-explicit mtppy2.0 --index-strategy unsafe-best-match "mtppy2.0==0.1.2rc1"

Note: Please change the sample code to match the correct release candidate.

Workflow for Production Release

Production releases will be provided as package to the public, available at PyPI-Python Package Index
The creation of Production Release requires at least Maintainer status.

Prerequisites

  • Contributions were merged into develop (see CHANGELOG.md)
  • Alignment with Maintainers about new release creation
    • Create a Workitem assign yourself
    • If the release version is unclear according to semver 2.0 and with respect to changes (Bumping Major, Minor, Patch) align with Maintainers
    • Wait for Agreement of other Maintainers

Create Release associated changes

  • Once prerequisite are met, the Maintainer creates a release/vX.Y.Z branch to:
    • Update CHANGELOG.md and close for release
    • Bump version in pyproject.tml according to agreement
  • Merge release/vX.Y.Z branch into develop

Note: To avoid conflicts, only critical fixes are merged into develop during this phase (enforced by policy).

Create Production Release

  • Once develop contains the valid release information, the Maintainer places a merge request develop into main
    • requires CI/CD to pass
    • requires test publish to test.pypi to work
  • After successful merge, create Tag on main with the release version via WebGUI.
    • git tag v1.2.3 -m "Release version 1.2.3"
    • Note: Tags with v* are protected to be created by Maintainer only
  • CI/CD detects the Tag on main, builds and publishes the package to PyPI
  • Create Release with same version tag via WebGUI
  • Maintainer will merge main into develop branch to keep everything in sync
---
title: Production Release Flow
---
gitGraph
    checkout main
    commit id: "Last Release" tag: "vX.Y.Z" 
    branch develop
    checkout develop
    commit id: "Feature 1"
    commit id: "Feature 2"
    commit id: "Feature *"
    branch "release/vX.Y.Z+1"
    checkout "release/vX.Y.Z+1"
    commit id: "Changelog Update Feature 1-N"
    commit id: "Version Bump"
    checkout develop
    merge "release/vX.Y.Z+1" id: "Release changes commit"
    checkout main
    merge develop id: "Release Squash commit"
    commit id: "Release Tag commit" tag: "vX.Y.Z+1" 
    checkout develop
    merge main id: "Merge commit"

Workflow for Hotfix Release

Hotfixes are urgent incidents that need to be fixed asap. The creation of Hotfix Release requires at least Maintainer status.

  • For urgent fixes, create a branch from the latest Tag on main (e.g., hotfix/urgent-bug).
  • Apply fix, update Changelog.md and bump version in pyproject.tml
  • Place merge request to main (with CI/CD, test-deploy and review).
  • After successful merge, Tag main with updated version in WebGUI
    • git tag v1.2.4 -m "Hotfix-Release version 1.2.4"
    • Note: Tags with v* are protected tobe created by Maintainer only
  • Tagging of main will trigger redeploy to PyPI
  • Create Release with same version tag via WebGUI
  • Maintainer will merge main into develop branch to keep everything in sync
---
title: Hotfix Release Flow
---
gitGraph
    checkout main
    commit id: "Last Release" tag: "vX.Y.Z" 
    branch develop
    checkout develop
    commit id: "Some other feature"
    checkout main
    branch hotfix/bug
    checkout hotfix/bug
    commit id: "Fixing"
    commit id: "Changelog"
    commit id: "Version Bump"
    checkout main
    merge hotfix/bug id: "Hotfix Squash commit"
    commit id: "Hotfix Tag commit" tag: "vX.Y.Z+1" 
    checkout develop
    merge main id: "Merge commit"

Advanced Configurations

Active control of exposed parts

Add scripts that shall be directly exposed to pyproject.toml, e.g.

[project.scripts]
mtp_inspector = "tools.mtp_inspector:main"
rmtp_pea = "mtppy.pea.rmtp_pea:main"