Architecting Out Loud
Back to posts
Cross-Repo Integration Testing · Part 2 of 2architecturegithub-actionsautomationdevops

Automating Cross-Repo Integration Testing with GitHub Actions

Jan 20, 20266 min read

Part 2 of 2. Part 1: 134 Packages, 3 Monorepos, Zero Guarantees covers the problem and the integration hub concept.

Part 1 ended with a concept: a dedicated integration hub repo (integration-hub), git submodules as version pins, and a known-good stack pointer that only advances when E2E passes. Good idea. But someone still had to update the submodules, open the PR, watch the tests, and hit merge. That's not a system, that's a chore list.

This is Part 2: making it run itself.

Three Workflows

The automation breaks into three GitHub Actions workflows. Each does one thing.

  1. Auto-Update: Detects new commits in source repos, proposes a new candidate combination
  2. Test: Runs the full E2E suite against the candidate
  3. Auto-Merge: Merges the candidate if tests pass, handles failures if they don't

A change in any source repo flows through all three automatically.

The Rolling PR Pattern

When a source repo pushes to main, the integration hub needs to test the new commit. The naive approach: open a new PR for every update. With three repos shipping independently, that creates a flood of PRs and stale ones that need cleanup.

Instead, we use a single rolling PR. One branch (auto/submodule-update), one PR, continuously updated.

The auto-update workflow runs on a schedule (every 6 hours) and on repository_dispatch events from the source repos. It pulls the latest main from all three submodules, commits the updated pointers, and force-pushes to the rolling branch. If the PR doesn't exist, it creates one. If it does, the push updates it in place.

One PR instead of many. Less noise. Multiple submodule updates batched together. A single place to see whether integration is currently passing.

Cross-Repo Triggers

When a developer merges a PR in any source repo, the integration hub should test immediately, not wait for the next 6-hour cron.

Each source repo has a small notify-e2e.yml workflow that fires on push to main. It sends a repository_dispatch event to the integration hub:

- name: Notify integration-hub
  uses: peter-evans/repository-dispatch@v3
  with:
    token: ${{ secrets.GH_SUBMODULE_PAT }}
    repository: ${{ github.repository_owner }}/integration-hub
    event-type: submodule-updated
    client-payload: |
      {
        "source_repo": "${{ github.repository }}",
        "sha": "${{ github.sha }}",
        "ref": "${{ github.ref }}"
      }

Fire-and-forget. The source repo's workflow finishes immediately. It doesn't wait for E2E to complete. Source repos stay fast and independent.

The integration hub's auto-update workflow picks up the dispatch, updates all submodules, and pushes to the rolling PR. The test workflow runs automatically on the PR.

The PR Lifecycle State Machine

The rolling PR moves through a set of states. The auto-merge workflow handles each one:

stateDiagram-v2
    [*] --> PR_Open: Auto-update creates/updates PR

    PR_Open --> Testing: Tests triggered on PR

    Testing --> Tests_Passed: All E2E pass
    Testing --> Tests_Failed: E2E failure

    Tests_Passed --> Check_Mergeable: Auto-merge checks PR state

    Check_Mergeable --> Merge: Clean, squash merge
    Check_Mergeable --> Behind_Main: PR is behind main
    Check_Mergeable --> Conflict: Merge conflict

    Behind_Main --> Testing: Update branch, re-test
    Conflict --> Alert: Slack notification, manual fix needed
    Tests_Failed --> Alert: Slack notification

    Merge --> [*]: Stack pointer advanced
    Alert --> PR_Open: Fix lands, auto-update runs again

This is a state machine, not a linear pipeline. The PR can cycle between states, going from "behind main" back to "testing" multiple times before eventually merging or alerting. In a busy repo, that loop is the normal case, not the edge case.

Failure Modes

Three things can go wrong. Each is handled differently. All Slack notifications originate from the auto-merge workflow (auto-merge-submodules.yml), the single point that evaluates PR check results and acts on them.

  • Tests fail. Auto-merge workflow detects the failing checks and sends a Slack message with the PR link, the failing run link, and which repos were in the candidate. Stack pointer doesn't advance. Source repos keep shipping (they aren't blocked). When a fix lands in any source repo, auto-update proposes a new candidate and the cycle restarts.
  • PR is behind main. Happens when main advances while tests are running (someone merged something else). Auto-merge calls GitHub's "update branch" API to merge main into the PR. This triggers a new test run. If that passes, it merges. Can loop a few times in high-traffic repos.
  • Merge conflict. PR can't be automatically merged. Auto-merge workflow sends a Slack message with the PR link and conflict details. Manual intervention required, typically rebasing the rolling branch. Rare, since the branch only contains submodule pointer updates.

The Happy Path

When everything works, the full sequence:

sequenceDiagram
    participant Source as Source Repo
    participant Hub as Integration Hub
    participant CI as E2E Tests
    participant Main as Hub Main

    Source->>Hub: Push to main (repository_dispatch)
    Hub->>Hub: Update submodule pointers, push rolling PR
    Hub->>CI: PR triggers full E2E suite
    CI->>Hub: Tests pass, automerge dispatched
    Hub->>Main: Squash merge PR
    Note over Main: Stack pointer advanced

From push in a source repo to merged stack pointer, no manual steps on the happy path.

Practical Notes

The default GITHUB_TOKEN can't trigger workflows in other repos or access private submodules. You need a PAT with repo + workflow scope. This is the #1 setup gotcha.

  • Secrets: A single PAT (GH_SUBMODULE_PAT) with repo + workflow scope, shared across the hub and source repos. The hub uses it for auto-update and auto-merge; source repos use it for the dispatch trigger.
  • GitHub Free tier: No branch protection rules or native auto-merge. The auto-merge workflow implements merge logic manually via the GitHub API, checking mergeable state, updating branches, and performing squash merges through API calls.
  • Race conditions: Two source repos push at nearly the same time. Both trigger dispatches. The second auto-update overwrites the first with a superset of changes (both updates). The rolling PR pattern handles this by design.
  • Manual override: The test workflow supports workflow_dispatch with ref inputs. You can manually test any combination of branches, tags, or SHAs from the GitHub Actions UI without going through the auto-update flow.

Takeaway

Three workflows. One rolling PR. A state machine that handles success, failure, staleness, and conflicts.

The integration hub doesn't slow down any source repo. It doesn't add required checks to source PRs. It runs in parallel, testing combinations as they appear, advancing the known-good stack pointer when they pass, and alerting when they don't.

We're not claiming this is the only way to do it. But 134 packages across 3 monorepos are now tested automatically, and we stopped finding out about integration failures during demos.

Resources