Troubleshooting GitHub Actions: Runner Out of Disk Space on Ubuntu 22.04 LTS Builds

Resolve 'runner out of disk space' errors in GitHub Actions on Ubuntu 22.04 LTS. Optimize workflows, manage caches, and reduce build artifacts for seamless CI/CD pipelines.

When your GitHub Actions workflow encounters a "runner out of disk space" error, it signifies that the virtual machine allocated for your build, typically an Ubuntu 22.04 LTS instance, has exhausted its available storage. This critical issue prevents your build, test, or deployment steps from completing, leading to failed CI/CD pipelines and disruption in your development workflow. This guide provides a highly technical, step-by-step approach to diagnose and resolve these disk space constraints.

Symptom & Error Signature

The most prominent symptom is a failed GitHub Actions workflow run. You'll observe log messages indicating a lack of available disk space, often manifested as commands failing with "No space left on device" errors or similar.

Here are typical log snippets you might encounter:

##[error]The runner has run out of disk space.
##[error]No space left on device

Error: fatal: write error: No space left on device
fatal: index-pack failed

npm ERR! code ENOSPC
npm ERR! syscall write
npm ERR! path /home/runner/.npm/_cacache/content-v2/sha512/...
npm ERR! errno -28
npm ERR! ENOSPC: no space left on device, write '/home/runner/.npm/_cacache/content-v2/sha512/...'
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/runner/.npm/_logs/....log

Error: Process completed with exit code 1.

If you add a step to inspect disk usage, you might see output like this:

df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           6.3G  1.2M  6.3G   1% /run
/dev/sda1        80G   80G     0 100% /
tmpfs            32G     0   32G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            32G     0   32G   0% /sys/fs/cgroup
/dev/sda15      105M  6.1M   99M   6% /boot/efi
tmpfs           6.3G  4.0K  6.3G   1% /run/user/1001

Notice Use% 100% for /dev/sda1 which is the root partition where the build workspace (/home/runner/work) resides.

Root Cause Analysis

GitHub-hosted runners, while powerful, come with finite resources. The Ubuntu 22.04 LTS runners typically offer ~80-100GB of disk space. When this limit is reached, it's usually due to one or a combination of the following factors:

1. Excessive Dependencies: Large node_modules directories (common in JavaScript projects), extensive Python virtual environments (venv), or massive Ruby gemsets can quickly consume gigabytes.
2. Unoptimized actions/checkout: By default, actions/checkout fetches the entire Git history. For large repositories with many commits or binary files, this can be substantial.
3. Build Artifacts and Intermediate Files: Compiling complex projects, especially C++, Java, or Go, can generate a vast number of object files, temporary compilation units, and large binaries. These often aren't cleaned up after their specific build step.
4. Inefficient Caching (actions/cache): While crucial for performance, improperly configured caches can lead to storing too much data, or caching items that change frequently, making the cache ineffective and bloated. Caches might also be restored and then not used, or even worse, grow without bound.
5. Docker Image Layers: Building Docker images involves downloading base images and creating multiple layers. If not managed carefully (e.g., without multi-stage builds or proper .dockerignore), the build context and intermediate layers can consume significant space. Subsequent Docker builds without proper cache management can re-download/re-build layers.
6. Temporary Directories: Unmanaged temporary files created by build tools or scripts in /tmp or within the workspace.
7. Cumulative Effect: Even if no single item is massive, a combination of many moderately sized files and directories can eventually fill the disk.

Step-by-Step Resolution

The key to resolving disk space issues is to be proactive in managing the runner's workspace and optimizing your workflow steps for efficiency.

1. Diagnose Disk Usage During Workflow Execution

The first step is to pinpoint what is consuming the disk space. Add diagnostic steps to your workflow.

jobs:
  build:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Initial disk space check
        run: |
          echo "--- Disk Space Before Build ---"
          df -h
          echo "--- Top 10 Largest Directories in Workspace ---"
          sudo du -h -d 1 /home/runner/work | sort -rh | head -10
          echo "--- Top 10 Largest Files in Workspace ---"
          sudo find /home/runner/work -type f -exec du -h {} + | sort -rh | head -10

      # Your existing build steps go here
      - name: Install dependencies
        run: npm ci

      - name: Build project
        run: npm run build

      - name: Final disk space check
        if: always() # Run even if previous steps fail
        run: |
          echo "--- Disk Space After Build ---"
          df -h
          echo "--- Top 10 Largest Directories in Workspace ---"
          sudo du -h -d 1 /home/runner/work | sort -rh | head -10
          echo "--- Top 10 Largest Files in Workspace ---"
          sudo find /home/runner/work -type f -exec du -h {} + | sort -rh | head -10

Analyze the output to identify rapidly growing directories or large files. Common culprits are /home/runner/work/<repo>/<repo>/node_modules, /home/runner/.npm, /tmp, or large build output directories.

2. Optimize actions/checkout Depth

For most CI/CD builds, you only need the latest commit, not the entire history. Reducing the fetch depth significantly reduces the size of the .git directory.

jobs:
  build:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout repository with shallow fetch
        uses: actions/checkout@v4
        with:
          fetch-depth: 1 # Only fetches the latest commit
          # For pull requests, you might need:
          # fetch-depth: 2 # Fetches the current commit and its parent for merge base calculations

3. Implement Strategic Caching with actions/cache

Utilize actions/cache to store and restore dependencies, but be selective about what you cache and ensure proper invalidation.

jobs:
  build:
    runs-on: ubuntu-22.04
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 1

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Cache Node.js modules
        uses: actions/cache@v4
        id: npm-cache # Give the step an ID to check for cache hit/miss
        with:
          path: ~/.npm # Cache the npm cache directory
          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
          restore-keys: |
            ${{ runner.os }}-node-

      - name: Install dependencies
        # Only install if cache was not hit, or if package-lock.json changed
        run: npm ci --prefer-offline
        if: steps.npm-cache.outputs.cache-hit != 'true'

      - name: Build application
        run: npm run build

• Cache Specific Directories: Don't cache entire project folders. Target specific dependency caches like ~/.npm, ~/.cache/pip, ~/.m2/repository.
• Keys: Use keys that invalidate when relevant files (e.g., package-lock.json, requirements.txt) change.
• Restore Keys: Use restore-keys for graceful fallback to slightly older caches.
• Conditional Installation: Only run dependency installation if the cache wasn't hit (if: steps.<cache-id>.outputs.cache-hit != 'true').

4. Clean Up Intermediate Files and Build Artifacts

After steps that generate large temporary files or build outputs, explicitly delete them if they are not needed for subsequent steps in the same job.

jobs:
  build:
    runs-on: ubuntu-22.04
    steps:
      # ... previous steps ...

      - name: Generate large report
        run: python generate_report.py > large_report.pdf

      - name: Upload report artifact (if needed)
        uses: actions/upload-artifact@v4
        with:
          name: report
          path: large_report.pdf
          retention-days: 7 # Keep for a week

      - name: Clean up large report
        if: always() # Ensure cleanup even if previous steps fail
        run: rm -f large_report.pdf

      - name: Build application
        run: |
          npm ci
          npm run build
          # Remove node_modules if not needed for subsequent steps in this job
          rm -rf node_modules
          # Remove intermediate build directories (e.g., for C++ or Java)
          rm -rf build/temp-obj/ generated-sources/

Use rm -rf with extreme caution. Always verify the paths you are deleting to ensure you don't remove critical files or the entire repository. Consider using find with maxdepth and delete for more precise cleanups.

5. Optimize Docker Builds

Docker builds are notorious for consuming disk space.

• .dockerignore: Ensure your .dockerignore file is comprehensive, excluding all unnecessary files (e.g., node_modules, .git, .vscode, tmp/) from the Docker build context.
• Multi-Stage Builds: Use multi-stage Docker builds to reduce the final image size and avoid including build-time dependencies in the production image.
• Docker Layer Caching: Leverage docker/build-push-action with GitHub Actions caching for Docker layers.

jobs:
  build-docker:
    runs-on: ubuntu-22.04
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Docker Hub (if pushing)
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Build and push Docker image
        uses: docker/build-push-action@v5
        with:
          context: .
          file: ./Dockerfile
          push: false # Set to true to push to registry
          tags: myapp:latest
          cache-from: type=gha # Restore cache from GitHub Actions cache
          cache-to: type=gha,mode=max # Store cache to GitHub Actions cache
          # Use specific target stage if using multi-stage builds
          # target: production-stage
          # arguments: # Pass build arguments, if any
          #   NODE_VERSION: "20"

      - name: Clean Docker system cache
        # This step removes unused Docker images, containers, volumes, and networks.
        # It's highly effective but can be aggressive.
        run: sudo docker system prune -a -f --volumes
        if: always() # Ensure this runs even if Docker build fails

docker system prune -a -f --volumes is a powerful command that will free up a lot of space, but it will remove all unused Docker objects. If you have multiple Docker-related steps or expect some images/volumes to persist across different steps within the same job (unlikely on a fresh runner, but good to note), consider its implications. For most GitHub Actions scenarios, it's safe and recommended as a cleanup step.

6. Utilize actions/upload-artifact and actions/download-artifact Judiciously

If artifacts are needed between jobs or for download after the workflow, use these actions. Be mindful of their size.

jobs:
  build:
    runs-on: ubuntu-22.04
    steps:
      # ... build steps ...
      - name: Build final artifact
        run: make package

      - name: Upload artifact
        uses: actions/upload-artifact@v4
        with:
          name: my-app-package
          path: ./dist/my-app.tar.gz
          retention-days: 7 # Optional: define how long to keep the artifact

Upload only the final, necessary artifacts. Avoid uploading entire build directories or temporary files.

7. Consider Self-Hosted Runners (Last Resort)

If your project's resource requirements consistently exceed what GitHub-hosted runners can provide, even after extensive optimization, consider using a self-hosted runner.

# In your workflow YAML
jobs:
  heavy_build:
    runs-on: self-hosted # Use your own machine or VM
    steps:
      # ... your resource-intensive steps ...

Self-hosted runners offer complete control over hardware (CPU, RAM, disk space), pre-installed software, and network configuration. However, they come with significant operational overhead, including maintenance, security updates, and ensuring high availability. This should be considered a last resort when all other optimization strategies for GitHub-hosted runners have been exhausted.

8. Reduce Build Matrix Complexity

If your workflow uses a matrix strategy (strategy.matrix) to build across many combinations (e.g., multiple OS versions, Node.js versions, Python versions), ensure each individual job in the matrix doesn't independently hit disk limits. If all combinations are run on separate runners, the disk space issue would be isolated to a single job, but if jobs share resources (less common for GitHub-hosted runners unless they are the same runner type but configured differently) or if one job's output affects another by using the same workspace path (highly unlikely), then a wider disk issue could occur. More likely, the problem is within a single matrix job.

By systematically applying these strategies, you can effectively manage disk space on GitHub Actions Ubuntu 22.04 LTS runners, ensuring reliable and efficient CI/CD pipelines.