Troubleshooting GitLab CI Runner: Resolving ‘Jobs Pending’ with Stuck Registration Token on WSL2 Ubuntu

Fix GitLab CI runner jobs stuck in pending on WSL2 Ubuntu due to registration token issues. Diagnose network, configuration, and WSL2-specific problems for seamless CI/CD.

Fix GitLab CI runner jobs stuck in pending on WSL2 Ubuntu due to registration token issues. Diagnose network, configuration, and WSL2-specific problems for seamless CI/CD.

GitLab CI/CD is a powerful tool for automating your software development lifecycle. However, setting up GitLab CI Runners in specific environments like Windows Subsystem for Linux 2 (WSL2) with an Ubuntu distribution can sometimes lead to unexpected hurdles. One common frustration is when your CI jobs remain in a "pending" state, and upon investigation, you discover the runner registration process is stalled or failing, often related to the registration token. This guide will walk you through diagnosing and resolving these issues, ensuring your CI jobs execute as expected on your WSL2-based runner.

Symptom & Error Signature

When your GitLab CI jobs are stuck in "pending" status, it indicates that no eligible runner is available or able to pick up the job. If you've recently attempted to register a new runner or suspect an existing one is misconfigured, you might observe the following:

  • GitLab UI: Jobs show "Pending" status indefinitely. The "Runners" section of your project/group might show your WSL2 runner as "Never contacted" or "Disconnected."

  • During gitlab-runner register command: The command might hang indefinitely after prompting for the registration token and executor, or it might fail with network-related errors.

    # Example of hanging registration
root@your-wsl2-ubuntu:~# gitlab-runner register
Runtime platform                                    arch=amd64 os=linux pid=1234 revision=abcdefg version=16.x.y
Running in system-mode.

Enter the GitLab instance URL (for example, https://gitlab.com/):
https://your-gitlab-instance.com/
Enter the registration token:
GR1349_YOUR_REGISTRATION_TOKEN_HERE
Enter a description for the runner:
My WSL2 Ubuntu Runner
Enter tags for the runner (comma-separated):
wsl2,ubuntu,docker
Enter an executor: docker, kubernetes, ssh, virtualbox, custom, shell:
docker
Enter the default Docker image (for example, ruby:2.7):
ubuntu:latest
Runner registered successfully. Feel free to start a job.
^C # You might have to Ctrl+C here if it hangs even after "Runner registered successfully."

# Example of a network-related registration failure
root@your-wsl2-ubuntu:~# gitlab-runner register
Runtime platform                                    arch=amd64 os=linux pid=1234 revision=abcdefg version=16.x.y
Running in system-mode.

Enter the GitLab instance URL (for example, https://gitlab.com/):
https://your-gitlab-instance.com/
Enter the registration token:
GR1349_YOUR_REGISTRATION_TOKEN_HERE
ERROR: Failed to register runner. A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

  • When checking runner status or logs:

    # Verify runner status
root@your-wsl2-ubuntu:~# systemctl status gitlab-runner
# Expected output: active (running)
# Actual output: potentially inactive, or active but with connection errors in logs

# Check runner logs for more details
root@your-wsl2-ubuntu:~# journalctl -u gitlab-runner -f
# Look for errors like:
# "Failed to dial GitLab: Get "https://your-gitlab-instance.com/api/v4/version": dial tcp 192.168.1.100:443: connect: connection refused"
# "Failed to send keep-alive packet"
# "x509: certificate signed by unknown authority"

Root Cause Analysis

The "jobs pending" and "stuck registration token" issue on a GitLab CI runner running within WSL2 Ubuntu often stems from a combination of network, configuration, and WSL2-specific integration challenges.

  1. Network Connectivity Issues:

    • Windows Firewall: The most common culprit. Windows Firewall on the host machine might be blocking outgoing connections from the WSL2 virtual network adapter to your GitLab instance (especially if it's self-hosted on a private IP or internal network) or inbound connections if GitLab needs to reach the runner (e.g., for specific webhook integrations, though less common for basic job polling).
    • DNS Resolution: WSL2 might have issues resolving the GitLab instance hostname, especially if it's an internal domain or using a custom DNS server.
    • Incorrect gitlab_url: A typo or incorrect protocol (HTTP vs. HTTPS) in the GitLab instance URL during registration or in config.toml.
    • Proxy Configuration: If you're behind a corporate proxy, the runner (and WSL2 environment) might not be correctly configured to use it.

  2. Incorrect Runner Configuration:

    • Invalid Registration Token: The token used might be expired, for a different project/group, or simply mistyped.
    • config.toml Errors: Malformed config.toml file, incorrect concurrent settings, or misconfigured executor parameters (e.g., pointing to a Docker socket that doesn't exist or isn't accessible).
    • SSL/TLS Certificate Issues: If your GitLab instance uses a self-signed certificate or one from an internal CA, the WSL2 Ubuntu environment might not trust it by default, leading to certificate validation errors.

  3. WSL2 Environment Specifics:

    • Docker Daemon Accessibility: If you're using the docker executor, the GitLab Runner needs to communicate with the Docker daemon. While Docker Desktop for Windows integrates well with WSL2, ensuring the Docker daemon is running and accessible from your specific WSL2 distribution is critical. Sometimes, the Docker socket path might be incorrect, or permissions might prevent access.
    • Systemd Integration: Older WSL2 setups might not have full Systemd support, meaning the gitlab-runner service might not start correctly or persist across reboots. Newer WSL2 versions (Windows 11) support Systemd natively.
    • Permissions: Incorrect file permissions on /etc/gitlab-runner/config.toml or the GitLab Runner binary itself can prevent it from operating correctly.

Step-by-Step Resolution

Follow these steps meticulously to diagnose and resolve your GitLab CI runner issues on WSL2 Ubuntu.

1. Verify Basic Network Connectivity from WSL2 to GitLab

First, ensure your WSL2 instance can reach your GitLab server.

  1. Open your WSL2 Ubuntu terminal.

  2. Ping your GitLab instance's hostname or IP address:

    ping your-gitlab-instance.com
# or if you know the IP
ping 192.168.1.100

    If ping fails, check DNS resolution and network routing.

  3. Check DNS Resolution:

    nslookup your-gitlab-instance.com

    If DNS fails, ensure your /etc/resolv.conf within WSL2 is correctly configured or that your Windows host's DNS settings are valid. WSL2 typically manages resolv.conf automatically.

  4. Test HTTPS connectivity to the GitLab API endpoint:

    curl -v https://your-gitlab-instance.com/api/v4/version

    Replace https://your-gitlab-instance.com/ with the exact URL of your GitLab instance. You should see an HTTP 200 OK response and JSON output with version information. If you get Could not resolve host, it's a DNS issue. If you get Connection refused or Failed to connect, it's a firewall or routing issue. If you see SSL certificate problem, then you have a certificate trust issue (see step 4).

2. Address Windows Firewall Rules

The Windows Firewall is a frequent blocker for WSL2 network traffic.

  1. Open Windows Firewall with Advanced Security. (Search for "Windows Firewall" in the Start Menu).

  2. Check Outbound Rules:

    • Ensure there are no rules explicitly blocking traffic from the vEthernet (WSL) adapter or your WSL2 IP range to your GitLab instance's IP and port (usually 443 for HTTPS).

  3. Check Inbound Rules:

    • While GitLab Runner primarily polls, if your GitLab instance is trying to connect to your runner for any reason (less common for basic CI), ensure port 80/443 or any custom ports aren't blocked.

  4. Create a specific outbound rule (if necessary):

    • Go to "Outbound Rules" -> "New Rule…"
    • Choose "Program" (if you want to target specific WSL2 executables, which is complex) or "Port" or "Custom".
    • For "Port", specify TCP and 443 (and 80 if applicable).
    • For "Scope", specify the remote IP address of your GitLab instance.
    • Allow the connection.
    • Apply to Domain, Private, and Public profiles as needed.

    While temporarily disabling the Windows Firewall can help diagnose if it's the culprit, never leave it disabled in a production or internet-facing environment. Always re-enable it and create specific, narrow rules.

3. Verify GitLab Runner Registration Details and Re-register

An incorrect URL or token is a common mistake.

  1. Retrieve a fresh registration token:

    • Go to your GitLab instance in your web browser.
    • Navigate to your project or group settings: Settings > CI/CD > Runners.
    • Locate the "Register a runner" section and copy the latest registration token.
    • Ensure the runner type (project/group/shared) and associated tags match your intent.

  2. Remove existing runner configuration (if any):

    • Stop the gitlab-runner service in WSL2:
      sudo systemctl stop gitlab-runner # For WSL2 with Systemd
# OR for older WSL2 without Systemd:
# sudo gitlab-runner stop
    • Delete the old configuration file and cache:
      sudo rm -f /etc/gitlab-runner/config.toml
sudo rm -rf /etc/gitlab-runner/certs/
    • From the GitLab UI, if you see the old runner listed, delete it to ensure a clean slate.

  3. Re-register the GitLab Runner:

    sudo gitlab-runner register
    • Enter the GitLab instance URL: https://your-gitlab-instance.com/ (double-check for typos!)
    • Enter the registration token: Paste the fresh token you copied.
    • Description & Tags: Provide meaningful details.
    • Executor: docker is highly recommended for flexibility.
    • Default Docker image: ubuntu:latest or alpine:latest are good general-purpose choices.

    If the registration command hangs or fails again, it points strongly to a network connectivity issue (Step 1 & 2) or SSL certificate issue (Step 4).

4. Address SSL/TLS Certificate Issues

If your GitLab instance uses a self-signed certificate or one from an internal Certificate Authority (CA), your WSL2 Ubuntu might not trust it.

  1. Download your GitLab instance's CA certificate:

    • Using your browser, access your GitLab instance, click on the padlock icon in the URL bar, and export the root CA certificate (typically in .pem or .crt format).
    • Alternatively, use openssl from a machine that trusts the certificate:
      echo -n | openssl s_client -showcerts -connect your-gitlab-instance.com:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > gitlab-ca.crt
    • Copy gitlab-ca.crt to your WSL2 Ubuntu instance (e.g., to /tmp/gitlab-ca.crt).

  2. Install the CA certificate in WSL2 Ubuntu:

    sudo cp /tmp/gitlab-ca.crt /usr/local/share/ca-certificates/gitlab-ca.crt
sudo update-ca-certificates

    You should see output indicating the certificate was added.

  3. Restart the GitLab Runner service:

    sudo systemctl restart gitlab-runner

    Or if Systemd is not running in WSL2:

    sudo gitlab-runner restart

5. Ensure Docker Daemon is Running and Accessible (for Docker Executor)

If you chose the docker executor, the runner needs to talk to the Docker daemon. Docker Desktop on Windows handles this integration, but verification is key.

  1. Ensure Docker Desktop for Windows is running on your Windows host.

  2. Verify Docker integration with WSL2:

    • In Docker Desktop settings, go to Resources > WSL Integration.
    • Ensure your Ubuntu distribution is enabled for Docker integration.

  3. Check Docker daemon status within WSL2 Ubuntu:

    docker info

    You should see detailed information about your Docker installation. If you see Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?, then Docker isn't correctly integrated or running.

    • Possible Fix: If docker info fails, try restarting Docker Desktop on Windows. Sometimes, restarting your entire Windows machine can resolve subtle WSL2 networking or Docker integration glitches.

  4. Verify config.toml Docker settings:

    • Open config.toml for editing:
      sudo nano /etc/gitlab-runner/config.toml
    • Look for the [[runners]] section and within it, the [runners.docker] subsection.
    • Ensure host = "unix:///var/run/docker.sock" or similar is correct. In a typical Docker Desktop + WSL2 setup, this should be handled automatically.

6. Start and Verify GitLab Runner Service

Finally, ensure the runner service is running and communicating with GitLab.

  1. Start the GitLab Runner service:

    • For WSL2 with Systemd (Windows 11+ or manually enabled):
      sudo systemctl start gitlab-runner
sudo systemctl enable gitlab-runner # Ensures it starts on WSL2 boot
sudo systemctl status gitlab-runner
    • For older WSL2 without Systemd:
      sudo gitlab-runner install
sudo gitlab-runner start
sudo gitlab-runner status

      If Systemd is not enabled, you'll need to manually start the runner after each WSL2 reboot or use a custom startup script. For Windows 11 users, enabling Systemd in WSL2 is highly recommended: Edit /etc/wsl.conf in your Ubuntu distribution:

      [boot]
systemd=true

      Then shut down WSL2 completely (wsl --shutdown in PowerShell) and restart your Ubuntu distro.

  2. Check runner logs for errors:

    sudo journalctl -u gitlab-runner -f # With Systemd
# OR
sudo tail -f /var/log/gitlab-runner.log # Without Systemd, if configured to log there

    Look for messages indicating successful connection to GitLab and job polling.

  3. Verify in GitLab UI:

    • Go to Settings > CI/CD > Runners in your GitLab project or group.
    • Your runner should now appear as "Connected" or "Online" (green circle) and show that it has contacted GitLab recently.

By systematically working through these steps, you should be able to resolve the "jobs pending" and "stuck registration token" issue for your GitLab CI runner on WSL2 Ubuntu, enabling your CI/CD pipeline to function smoothly.