Certbot DNS-01 Challenge Failure: Troubleshooting TXT Records Mismatch

Resolve 'Certbot DNS-01 challenge verification failed TXT records mismatch' errors. This guide details root causes and step-by-step fixes for failed Let's Encrypt SSL certificate renewals and issuances.

When issuing or renewing Let’s Encrypt SSL certificates using Certbot’s DNS-01 challenge method, encountering a “TXT records mismatch” error can be a frustrating roadblock. This issue prevents Certbot from verifying domain ownership, leading to failed certificate operations and potentially expired SSL certificates, causing browser warnings for your users. This guide provides a deep dive into the underlying causes and offers a structured, step-by-step resolution process for experienced system administrators and DevOps engineers.

Symptom & Error Signature

You will typically observe this error in your terminal output when attempting to run certbot certonly, certbot renew, or similar commands, especially when using a DNS plugin (e.g., certbot-dns-cloudflare, certbot-dns-route53). The output will indicate that the challenge could not be verified because the expected TXT record value does not match what Certbot found, or no record was found at all.

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator dns-cloudflare, Installer nginx

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert not due for renewal, but simulating renewal for dry run
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Renewing an existing certificate for example.com and www.example.com
Performing the following challenges:
dns-01 challenge for example.com
dns-01 challenge for www.example.com
Waiting for verification...
Challenge failed for domain example.com
Challenge failed for domain www.example.com
dns-01 challenge for example.com failed
dns-01 challenge for www.example.com failed
Cleaning up challenges
Failed to renew certificate example.com with error: Some challenges have failed.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: example.com
   Type:   dns
   Detail: DNS problem: NXDOMAIN looking up TXT for _acme-challenge.example.com -
   DNS problem: SERVFAIL looking up TXT for _acme-challenge.example.com -
   DNS problem: REFUSED looking up TXT for _acme-challenge.example.com -
   DNS problem: query timed out looking up TXT for _acme-challenge.example.com -
   DNS problem: CNAME record found for _acme-challenge.example.com
   DNS problem: The key authorization was not found or has an incorrect value.
   Expected: "SOME_EXPECTED_CHALLENGE_STRING_FROM_LE"
   Got:      "SOME_INCORRECT_OR_OLD_CHALLENGE_STRING" (order 1 of 1)
   (Caused by: [('timed out',), ('NXDOMAIN',), ('SERVFAIL',), ('REFUSED',), ('CNAME',)])

   Domain: www.example.com
   Type:   dns
   Detail: The TXT record "_acme-challenge.www.example.com" does not match the
   expected value. Expected: "ANOTHER_EXPECTED_CHALLENGE_STRING_FROM_LE".
   Got: "ANOTHER_INCORRECT_OR_OLD_CHALLENGE_STRING"

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Root Cause Analysis

The DNS-01 challenge relies on Certbot instructing you (or its DNS plugin) to create a specific TXT record containing a unique challenge token at _acme-challenge.yourdomain.com. Let’s Encrypt’s servers then query the DNS for this record. If the record is found and its value matches the expected token, domain ownership is verified, and the certificate is issued or renewed.

A “TXT records mismatch” error indicates that Let’s Encrypt could not find the correct TXT record at the expected DNS name (_acme-challenge.yourdomain.com). This can stem from several underlying issues:

  1. DNS Propagation Delays: This is the most common cause. Changes made to DNS records, especially new ones, take time to propagate across the global DNS infrastructure. If Certbot verifies too quickly, Let’s Encrypt’s resolvers may query a DNS server that hasn’t yet updated.
  2. Incorrect TXT Record Value:
    • Typos or extra characters: Simple human error when manually entering the value, or an issue with the DNS plugin.
    • Leading/Trailing Spaces: Many DNS providers will silently strip these, but some may not, causing a mismatch.
    • Incorrect Encoding: Rare, but possible with certain systems.
    • Old Record Present: If a previous challenge failed, an old _acme-challenge TXT record might still exist, causing a conflict with the new one.
  3. Incorrect TXT Record Name:
    • Missing _acme-challenge prefix: The record must be specifically _acme-challenge.yourdomain.com (or _acme-challenge.subdomain.yourdomain.com).
    • Wrong Subdomain: If renewing for www.example.com, the challenge record must be _acme-challenge.www.example.com, not _acme-challenge.example.com.
    • Provider-Specific Naming: Some DNS providers might have slightly different conventions (e.g., only expecting _acme-challenge in the “Name” field, and appending the domain automatically).
  4. Multiple Competing TXT Records: If you have more than one TXT record for _acme-challenge.yourdomain.com, Let’s Encrypt might pick an incorrect one, or your DNS provider might handle multiple records differently.
  5. DNS Provider API Issues (for plugin users):
    • API Key/Secret Permissions: The credentials provided to Certbot’s DNS plugin might lack the necessary permissions to create, update, or delete TXT records.
    • API Rate Limits: The DNS provider might be temporarily throttling API requests.
    • Transient Provider Errors: Temporary outages or service degradation at the DNS provider.
  6. Caching: Your local DNS resolver cache, or your ISP’s recursive DNS cache, might be serving stale information.
  7. CNAME Interference: If _acme-challenge.yourdomain.com is itself a CNAME record pointing to another domain, the TXT record needs to be on the target of the CNAME, not directly on _acme-challenge.yourdomain.com.

Step-by-Step Resolution

Follow these steps meticulously to diagnose and resolve the Certbot DNS-01 challenge failure.

1. Verify the TXT Record Manually

The first step is always to verify the actual state of your DNS records using independent tools.

  1. Identify the expected challenge string: Rerun Certbot with --dry-run and --debug-challenges (if available and not using a plugin that cleans up immediately) or observe the error message carefully. The error message usually tells you the Expected: "..." value.

    sudo certbot renew --dry-run -vvv
# Or for a new cert:
sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials ~/.secrets/cloudflare.ini -d example.com -d www.example.com --dry-run -vvv

    Look for lines like _acme-challenge.example.com TXT "..." which might indicate what Certbot is trying to set. If you’re using a DNS plugin, Certbot will usually try to set it and then fail. The error message itself is often the best source for the expected value.

  2. Construct the full TXT record name: For example.com, it’s _acme-challenge.example.com. For www.example.com, it’s _acme-challenge.www.example.com.

  3. Query DNS using dig or nslookup:

    # For Linux/macOS
dig TXT _acme-challenge.example.com +short

# For Windows
nslookup -type=TXT _acme-challenge.example.com

    [!TIP] Querying from different DNS resolvers (e.g., Google’s 8.8.8.8 or Cloudflare’s 1.1.1.1) can help confirm global propagation. dig TXT _acme-challenge.example.com @8.8.8.8

  4. Compare “Got” vs. “Expected”:

    • If dig returns an empty result, the record might not exist or hasn’t propagated.
    • If dig returns a value, compare it exactly to the “Expected” value from Certbot’s error output. Pay close attention to case sensitivity (though TXT record values are usually case-sensitive, domain names are not), leading/trailing spaces, and any special characters.

2. Account for DNS Propagation Delays

DNS changes take time. The TTL (Time To Live) of your DNS records dictates how long recursive DNS servers should cache a record. Even if you set a very low TTL (e.g., 60-300 seconds), it can still take a few minutes up to an hour for changes to fully propagate worldwide, especially if older TTL values were high.

  1. Wait: After creating or modifying the TXT record at your DNS provider, wait for at least 5-15 minutes, or even longer if your domain previously had a high TTL.
  2. Retry Certbot: After waiting, attempt the Certbot command again. 
    sudo certbot renew
# Or, if it's a new certificate issuance
sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials ~/.secrets/cloudflare.ini -d example.com -d www.example.com
  3. Automated Delay (for scripting): If you’re scripting Certbot calls with a DNS provider’s API, ensure your script includes a sufficient sleep interval after creating the record before triggering Certbot’s verification. Many Certbot DNS plugins handle this automatically, but custom scripts might not.

3. Inspect DNS Provider Configuration

Thoroughly review your DNS provider’s interface for the specific domain.

  1. Record Name Accuracy: Ensure the “Name” or “Host” field for the TXT record is exactly _acme-challenge (or _acme-challenge.subdomain if applicable). Most providers will automatically append your domain name.

    [!IMPORTANT] Do NOT enter _acme-challenge.yourdomain.com in the “Name” field unless your DNS provider specifically instructs you to. Typically, you only enter the subdomain part, _acme-challenge.

  2. Record Value Accuracy: Double-check the value.

    • No leading or trailing spaces.
    • Exact match, including case.
    • Ensure it’s the current expected value, not an old one.

  3. One TXT Record Only: Verify that there is only one TXT record for _acme-challenge.example.com. If multiple exist, delete the outdated or incorrect ones.

  4. Check for Other Services: Some services (e.g., Mailgun, SPF records) might also use TXT records. Ensure there’s no conflict, although _acme-challenge is generally unique to Let’s Encrypt.

  5. DNS Provider API Credentials (if using plugins):

    • Location: Ensure your API credentials file (e.g., ~/.secrets/cloudflare.ini, /etc/letsencrypt/secrets/route53.ini) is correctly configured and has the correct permissions.
    • Permissions: Verify that the API key/token has the necessary permissions to modify TXT records for the specific domain. For example, Cloudflare API tokens need “Zone DNS:Edit” permission.
    • Environment Variables: If credentials are passed via environment variables, ensure they are correctly set in the shell where Certbot runs.
    # Example: ~/.secrets/cloudflare.ini
dns_cloudflare_api_token = YOUR_CLOUDFLARE_API_TOKEN
    sudo chown root:root ~/.secrets/cloudflare.ini
sudo chmod 600 ~/.secrets/cloudflare.ini

4. Clear Local DNS Cache

If you’re repeatedly querying DNS and seeing old values, your local system’s DNS cache might be stale.

  1. Flush systemd-resolved cache (Ubuntu/Debian): 
    sudo systemd-resolve --flush-caches
sudo systemctl restart systemd-resolved
  2. Flush nscd cache (older systems or specific setups): 
    sudo systemctl restart nscd
  3. Flush browser DNS cache: If observing issues in a browser, clear its DNS cache.

5. Debug Certbot DNS Plugins

If you’re using a Certbot DNS plugin, enable verbose logging for more detailed insights.

  1. Run with Verbosity: Add -vvv to your Certbot command. This provides extensive debugging output.

    sudo certbot renew --dry-run -vvv

    Examine the logs in /var/log/letsencrypt/letsencrypt.log for specific API calls, responses from the DNS provider, and any errors reported by the plugin itself before the challenge verification step.

  2. Plugin-Specific Issues: Check the documentation for your specific Certbot DNS plugin. Some plugins might have unique configuration requirements or known issues.

6. Troubleshoot CNAME Interference

If dig TXT _acme-challenge.example.com returns a CNAME record instead of a TXT record, this is a problem.

  1. Identify CNAME: 
    dig CNAME _acme-challenge.example.com +short
    If it points to some.other.domain.com, then the TXT record for the ACME challenge must be placed at _acme-challenge.some.other.domain.com, not _acme-challenge.example.com.
  2. Remove or Correct CNAME: If the CNAME is unintentional or incorrect, remove it. If it’s intentional (e.g., for delegated ACME challenges), ensure the target domain properly hosts the TXT record.

7. Retrying Certbot

After making changes and allowing for propagation, retry Certbot.

  1. Standard Renewal/Issuance:

    sudo certbot renew --cert-name example.com
# Or for a new cert
sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials ~/.secrets/cloudflare.ini -d example.com -d www.example.com

  2. Forcing Renewal (if needed): If Certbot thinks the certificate is not yet due for renewal but you’ve fixed an underlying issue, you might use --force-renewal. Use this sparingly.

    sudo certbot renew --force-renewal

  3. Cleanup Previous Challenges: In rare cases, if old challenge files or records are causing persistent issues, you might need to clean up.

    • Manually delete old TXT records from your DNS provider.
    • Certbot’s DNS plugins are designed to clean up, but if a process was interrupted, manual cleanup might be required.

    [!WARNING] Be cautious with certbot delete. It will remove the entire certificate and its configuration, requiring a full re-issue. Only use this if you are absolutely sure you want to start fresh and have backups of your Nginx/Apache configuration if Certbot modified it.

By systematically working through these troubleshooting steps, you should be able to identify and resolve the “Certbot DNS-01 challenge verification failed TXT records mismatch” error and successfully secure your domains with Let’s Encrypt SSL certificates.