How Two Spaces Almost Sank an Aircraft Carrier: A Kamal Deployment Story

Your Rails app is tested. Your CI is green. Your Kamal config is pristine. But SCAN_RUBY found a tab character on line 47 and now the entire deployment is grounded. This is the story of how whitespace nearly ended civilization.

How Two Spaces Almost Sank an Aircraft Carrier: A Kamal Deployment Story

How Two Spaces Almost Sank an Aircraft Carrier: A Kamal Deployment Story

You've been there. It's 4:47 PM on a Friday. The feature is done. The tests pass. The client is refreshing their browser every eleven seconds waiting for the update. You type kamal deploy with the quiet confidence of someone who has done this a hundred times.

And then the terminal turns red.

Not because your code is broken. Not because your Docker image failed to build. Not because your server ran out of memory. No. Because SCAN_RUBY found a tab character on line 47 of a file you haven't touched since 2019.

Welcome to modern deployment, where the real enemy isn't downtime — it's whitespace.

The Linting Industrial Complex

Somewhere along the way, we collectively decided that deploying working software wasn't enough. The code also had to be beautiful. Not beautiful in the way that matters — readable, well-structured, logically organized — but beautiful in the way that a machine defines beauty: two spaces. Not three. Not one. Not a tab. Two spaces. Exactly two. Every time. Forever.

SCAN_RUBY doesn't care that your feature works. SCAN_RUBY doesn't care that your tests cover 97% of your codebase. SCAN_RUBY cares that on line 203 of app/services/invoice_generator.rb, you used hash rocket syntax instead of the new symbol syntax, and frankly, SCAN_RUBY is disappointed in you.

And SCAN_RUBY brought a friend. TRIVY is here too, and TRIVY has found a CVE from 2021 in a transitive dependency of a gem you use to parse CSV files that you imported once for a rake task that no one has run in eight months. TRIVY has classified this as CRITICAL. The deployment is halted. The aircraft carrier is dead in the water.

The Aircraft Carrier Scenario

Let's talk about aircraft carriers for a moment, because the analogy is perfect.

The USS Gerald R. Ford cost $13.3 billion to build. It carries 75 aircraft, has a crew of 4,539 people, and can project military force across an entire hemisphere. It is, by any reasonable measure, one of the most complex machines ever constructed by human beings.

Now imagine the USS Gerald Ford is ready to deploy. The jets are fueled. The crew is at battle stations. The admiral gives the order. And then — nothing. The ship doesn't move. Why?

Because an automated linting system scanned the ship's navigation software and found that Ensign Rodriguez used tabs instead of spaces in a configuration file for the backup coffee maker on deck 3.

"Sorry, Admiral. SCAN_NAVY found 847 style violations in the galley management subsystem. We can't launch until every single one is resolved. Also, TRIVY found that the firmware on torpedo bay door #7 depends on a version of OpenSSL that was flagged eighteen months ago. Yes, the torpedoes work. No, that doesn't matter."

The admiral stares at the terminal. The terminal stares back. Somewhere in the Pacific, the enemy is not running a linter.

Kamal: The Beautiful, Merciless Deployment Tool

Let's be clear: Kamal is excellent. It's what deployment should have been all along — simple, Docker-based, zero-downtime, no Kubernetes PhD required. DHH and the Rails team built something genuinely great. You define your servers, point at your Docker registry, and kamal deploy handles the rest. Rolling restarts, health checks, asset bridging, the works.

But Kamal, being a well-behaved member of the modern deployment ecosystem, respects your CI pipeline. And your CI pipeline — that beautiful, pristine, 47-step GitHub Actions workflow that you spent three days configuring — includes SCAN_RUBY and TRIVY. Because of course it does. Because someone on the team read an article about "shift-left security" and now every deploy has to pass through more checkpoints than a border crossing.

So here's what actually happens when you run kamal deploy:

  1. Docker builds your image ✅
  2. Tests pass ✅
  3. Assets compile ✅
  4. SCAN_RUBY wakes up from its slumber 🔴
  5. "Offense detected: Layout/IndentationWidth — Use 2 spaces for indentation, not 1"
  6. Your deploy is now a philosophical debate about whitespace
  7. The client emails asking if the site is down

The Two Spaces vs. Tabs Holy War

This particular religious conflict has been raging since approximately 1975, and like all truly important religious conflicts, both sides are absolutely certain they're right and the other side is morally deficient.

The spaces camp argues that spaces render identically everywhere. Two spaces is two spaces whether you're in VS Code, Vim, nano, or a government terminal from 1997. Consistency. Predictability. Order.

The tabs camp argues that tabs are semantically correct — they represent indentation, not a specific number of characters — and that tab width should be configurable per developer. Freedom. Flexibility. Self-expression.

And then there's the "I literally do not care, my code works and the client is waiting" camp, which is by far the largest group but has no representation in the linting community because linting tools are written by people in the first two camps.

SCAN_RUBY has chosen sides. SCAN_RUBY is a spaces partisan. And SCAN_RUBY has veto power over your deployments.

The TRIVY Situation

If SCAN_RUBY is the grammar teacher who fails your essay because you used a semicolon incorrectly, TRIVY is the building inspector who condemns your house because the screws in your mailbox are 2mm too short.

TRIVY scans your Docker image for known vulnerabilities. This is, in principle, a good thing. In practice, TRIVY will find vulnerabilities in:

  • Base images you don't control
  • System libraries installed by your OS layer
  • Transitive dependencies of transitive dependencies
  • Software that was deprecated before your company existed
  • The concept of time itself (CVE-2024-EPOCH: "Time continues to pass, causing all software to become older")

TRIVY doesn't distinguish between "a remote attacker could execute arbitrary code on your production server" and "a theoretical vulnerability exists in a library that's only loaded during test runs on machines running a specific version of FreeBSD." Both are CRITICAL. Both block your deploy. Both require you to stop what you're doing and investigate.

Your deploy pipeline now looks like this:

Step 1: Build (30 seconds)
Step 2: Test (2 minutes)
Step 3: SCAN_RUBY argues about aesthetics (45 seconds)
Step 4: TRIVY catalogues the entire history of computer security (4 minutes)
Step 5: You fix a whitespace issue from 2019 (5 minutes)
Step 6: You Google a CVE from 2022 (15 minutes)
Step 7: You update a gem that breaks three other gems (45 minutes)
Step 8: You fix the three other gems (2 hours)
Step 9: You run the pipeline again (7 minutes)
Step 10: TRIVY found a new CVE while you were fixing the old one
Step 11: Goto Step 6
Step 12: The heat death of the universe
Step 13: Deploy

The Real Cost

Here's where the sarcasm gives way to something almost serious.

Every hour spent fixing lint violations that have nothing to do with functionality is an hour not spent building features. Every deploy blocked by a low-severity CVE in a test dependency is a deploy that could have shipped value to users. Every developer who has to context-switch from "building the thing" to "satisfying the robot" loses 20-30 minutes of productive flow state.

I'm not arguing against code quality. I'm not arguing against security scanning. I'm arguing that the defaults are insane. A lint violation in a comment should not have the same blocking power as a failed test. A low-severity CVE in a dev dependency should not prevent production deployments.

The tools should serve the deployment. The deployment should not serve the tools.

What To Actually Do About It

If you're running Kamal deploys and getting blocked by SCAN_RUBY and TRIVY, here's the pragmatic approach:

For SCAN_RUBY / RuboCop:
- Set up a .rubocop.yml that reflects your team's actual preferences, not the gem's defaults
- Use rubocop --auto-correct to fix the easy stuff in bulk
- Disable cops that are purely cosmetic in CI (looking at you, Layout/IndentationWidth)
- Run linting as a non-blocking step — report violations, don't fail the build

For TRIVY:
- Use a .trivyignore file for known false positives and accepted risks
- Set severity thresholds — block on CRITICAL/HIGH, warn on MEDIUM/LOW
- Pin your base images to versions you've already scanned
- Run TRIVY on a schedule, not on every single deploy

For your sanity:
- Remember that shipping working software is the point
- A linter is a tool, not a moral authority
- Two spaces and tabs both compile to the same bytecode
- The aircraft carrier should probably just launch

Conclusion

Somewhere right now, a developer is staring at a terminal that says deploy failed. Their code works. Their tests pass. Their Docker image builds clean. But SCAN_RUBY found an extra blank line at the end of a file, and TRIVY discovered that the zlib version in the Alpine base image was flagged by a researcher in 2023 for a vulnerability that requires physical access to the server and a working knowledge of Klingon to exploit.

The aircraft carrier sits in port. The admiral is on hold with IT. Ensign Rodriguez is reformatting the coffee maker config with two spaces per indent, as God and the RuboCop maintainers intended.

The enemy has shipped six features while you were running bundle update.

Ship the code. Fix the lint later. The carrier needs to sail.


Loadout helps teams deploy Rails applications with Kamal, manage CI/CD pipelines, and occasionally argue about whitespace. If your deploys are stuck, we can help.