Why a Jenkins Migration Took Seven Months — and Three Others Did Not

We migrated four customers off Jenkins. Three went smoothly. The fourth took seven months. The difference wasn't the pipelines. It was the five things Jenkins kept hidden.

John Baek
John Baek
Founder, CollabOps
Why a Jenkins Migration Took Seven Months — and Three Others Did Not

In the last 18 months I've moved four customers off Jenkins. Three landed on schedule — six weeks, nine, eleven. The fourth took seven months.

The difference wasn't the pipeline complexity. The fourth customer's pipelines were the simplest of the four. The difference was the five things Jenkins had quietly stored outside the visible system.

1. Shared Library

Jenkins's Shared Library is internal Groovy DSL. Drop a file like vars/deployToK8s.groovy and every pipeline can call deployToK8s(env: 'prod').

The trap is that this is macros, not code. There's no static analysis. Which pipeline uses which macro is only visible at runtime.

The fourth customer had 47 shared library functions. We found 23 in the first month. By month seven we had found 41. The last six were called only by a quarterly nightly job — neither we nor the customer remembered they existed.

2. The plugin DAG

The real shape of Jenkins isn't the pipelines. It's the plugin dependency graph. An average enterprise Jenkins instance has 40–80 plugins. Of those:

  • Half — nobody remembers why
  • A third — no update in over a year
  • A quarter — expired free license or original maintainer disappeared

The fourth customer had 73 plugins. Eleven deprecated. Four were internal forks with no documentation. Migration meant analyzing all 73 and deciding what each maps to in the new system. That's not pipeline conversion. That's reverse engineering.

3. Credentials

Jenkins's credentials store is an encrypted XML tree. Humans can't read it directly; auto-exporting is possible — but the encryption key lives only on that instance.

The clock starts when migration begins: if the customer can't recall the exact issuance date of every credential, every credential has to be re-issued. The fourth customer had 119 credentials. They remembered the original issuer for seven of them. Security decided to rotate everything. That alone was two months.

4. The pipeline assumed an environment, not just code

Jenkins declarative pipelines look like code. They live in Jenkinsfile, in git.

pipeline {
    agent { label 'linux-arm64-fast' }   // ← defined where?
    environment {
        REGISTRY = "${REGISTRY_PROD}"     // ← global env, from where?
    }
    ...
}

Both lines depend on external state of the Jenkins instance. The agent label is defined somewhere, mapped to which machines? Is REGISTRY_PROD a system env var, a Jenkins-UI variable, or injected by a plugin?

Without reproducing this, no build ever runs the same way twice. The fourth customer's agent had a manually applied patch, and that patch turned out to be load-bearing for one specific build. Finding it: three weeks.

5. People's habits

This is the most expensive of the five. The Jenkins Replay button. The infinite-scroll console output on a build history page. The big red Build Now button on the dashboard. These habits become part of how the org operates.

Move to a new system and the operators have to do the same things in different places. That isn't training. It's muscle memory rewiring. The fourth customer had a 12-person ops team. Six adapted in three months. The other six took six. We had to run both systems in parallel for those six months.

What separated the four

The three smooth migrations had something in common — Jenkins had been running for under a year, and the same person had owned both the install and the migration. The hidden assumptions lived in that one person's head.

The fourth customer had run Jenkins for seven years and gone through three full rotations of ops staff. The assumptions had left with the people, with no documentation behind them.

How to avoid the seven-month version

Before deciding to migrate, audit five things:

  1. Shared library function usage frequency — including ones called once a year.
  2. Plugin DAG with deprecated / unmaintained flags.
  3. Whether each credential has a recoverable issuer record.
  4. Reproducibility of agent nodes — do they boot from an immutable image?
  5. Ops team tenure — median under 2 years means half the assumptions have already left.

All five clean → six to twelve weeks. Even one black box → quarter-scale work.

So when is the right time to leave Jenkins?

The right time to leave Jenkins is after running the five-point audit above (shared-library usage, plugin DAG, credential issuer records, agent reproducibility, ops-team tenure), not before. If you're an engineering lead evaluating the migration right now, the audit won't shrink seven months into six weeks. It will tell you which one you're signing up for. The most expensive thing is starting blind.

Tags#cicd#jenkins#migration#devops#infrastructure