Adopting Cosign Image Signing — *Key Rotation* Costs More Than Adoption Itself

Cosign adoption posts cover how to sign. That step takes two weeks. The next seven months covered what almost every guide omits — key rotation.

This post is the four assumptions that broke in those seven months.

Assumption 1 — keys in safe storage don't need rotation

Keys in a KMS are assumed not to leak, so no rotation needed. Wrong.

Why keys must rotate:

• KMS policy itself (AWS KMS auto-rotation, when enabled, mandates it)
• Key access changes (someone with key permission departs → rotation needed)
• Standards guidance (NIST SP 800-57: signing keys rotate on a multi-year cadence)
• Real leaks (KMS itself stays safe, but client-side leakage happens)

Assumption 2 — rotation = issue a new key

Most automation implementations interpret rotation as issue new key + sign with new key. Wrong.

Real rotation is five steps:

1. Issue the new key
2. Distribute the new key's public part to verifiers
3. Transition window — both old and new keys valid
4. Start signing with the new key only
5. Revoke the old key (rotate or delete)

Step 3, the transition window, is load-bearing. Skip it and all verifications fail right after rotation.

Assumption 3 — verifiers update instantly

Distribute the new key's public part to verifiers (admission controllers, scanners) and assume instant uptake. Wrong.

Each verifier has its own key cache TTL. Kyverno is 5 minutes; Sigstore Policy Controller is different; etc. Not instant. The transition window must be at least 2× the longest cache TTL.

Our case — we set the window to 15 minutes. We didn't realize one verifier's cache TTL was 24 hours. After rotation, images were rejected for 24 hours in some environments.

Assumption 4 — rotation cadence figures itself out

Rotation cadence isn't automatic. Organizations must decide explicitly.

Drivers:

• Legal requirements (FIPS, certain regulated industries) — annual is the standard
• Personnel turnover — if key holders change quarterly, rotate quarterly
• KMS billing — every rotation creates new KMS resources (you keep the old one too — still needed for verification)
• Post-incident response — rotate immediately on suspected leak

Our decision — annual + immediate on incident. Two months of political work to land on that.

Seven months — time allocation

Work                                          | Time    | Note
──────────────────────────────────────────────┼─────────┼─────────────────────
Cosign adoption (signing + verifying)         | 2 wk    | Per the guide
Rotation policy decision                      | 8 wk    | Heavy political cost
Rotation automation (5-step)                  | 4 wk    | All five steps
Verifier cache TTL discovery + alignment      | 4 wk    | Different per verifier
First rotation attempt + failure + postmortem | 4 wk    | 24-hour image rejection
Second rotation attempt + success             | 2 wk    | 
Rotation as a quarterly routine               | 4 wk    | 
──────────────────────────────────────────────┼─────────┼─────────────────────
Total                                          | 28 wk = 7 months

Cosign adoption itself was 7%. The other 93% was rotation.

Lesson from the failed rotation

The 24-hour rejection postmortem concluded — we must know every verifier's cache TTL. If new verifiers are registered without documenting it, the next rotation breaks.

We added a cache TTL field to verifier registration. Without that field, no new verifier registers.

Can you adopt Cosign signing now and defer key rotation?

Ship Cosign signing without key rotation and let six months pass, and you'll meet all four assumptions above at once on rotation day. For security and DevOps leads evaluating Cosign, or already running signature-only, the answer is to design rotation together with signing from day one.