Nginx Worker Process: Permission Denied Opening Unix Socket on Ubuntu 20.04 LTS

Troubleshoot 'Permission Denied' errors when Nginx worker processes fail to open Unix sockets on Ubuntu 20.04 LTS. Resolve common user, group, and directory permission issues.

When managing web servers on Ubuntu 20.04 LTS, encountering a "Permission denied" error from an Nginx worker process trying to access a Unix socket is a common yet often perplexing issue. This error typically prevents Nginx from communicating with backend processes, most frequently PHP-FPM, leading to a non-functional website, often displaying a "502 Bad Gateway" error to visitors or simply failing to start the Nginx service. This guide will walk you through diagnosing and resolving these permissions-related hurdles.

Symptom & Error Signature

The primary symptom is a website that fails to load content served by a backend application (like PHP), often showing a "502 Bad Gateway" error, or Nginx failing to start entirely. You'll find critical error messages in your Nginx error logs (typically /var/log/nginx/error.log) or within your systemd journal.

Typical Nginx Error Log Entry:

2023/10/26 14:30:05 [crit] 12345#12345: *1 connect() to unix:/var/run/php/php7.4-fpm.sock failed (13: Permission denied) while connecting to upstream, client: 192.168.1.100, server: example.com, request: "GET /index.php HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.4-fpm.sock:", host: "example.com"

Or, if Nginx itself is trying to open a custom socket:

2023/10/26 14:30:10 [emerg] 12345#12345: open() "/var/run/nginx/custom.sock" failed (13: Permission denied)

Systemd Journal Output (checking Nginx service status):

$ sudo systemctl status nginx
● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Mon 2023-10-26 14:30:15 UTC; 1s ago
    Process: 12345 ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;' (code=exited, status=1/FAILURE)
    Process: 12346 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry=TERM/5/KILL/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)
   Main PID: 12345 (code=exited, status=1/FAILURE)

Oct 26 14:30:15 hostname nginx[12345]: 2023/10/26 14:30:15 [emerg] 12345#12345: connect() to unix:/var/run/php/php7.4-fpm.sock failed (13: Permission denied) while connecting to upstream
Oct 26 14:30:15 hostname systemd[1]: nginx.service: Main process exited, code=exited, status=1/FAILURE
Oct 26 14:30:15 hostname systemd[1]: nginx.service: Failed with result 'exit-code'.
Oct 26 14:30:15 hostname systemd[1]: Failed to start A high performance web server and a reverse proxy server.

Root Cause Analysis

The "Permission denied" error (errno 13) indicates that the Nginx worker process, running under a specific user and group (typically www-data on Ubuntu), does not have the necessary read and write permissions to interact with the Unix domain socket file or its parent directory. This can stem from several underlying causes:

1. PHP-FPM Socket Permissions (Most Common):

   • User/Group Mismatch: The PHP-FPM process creates the .sock file, and its default listen.owner and listen.group settings in the FPM pool configuration might not grant access to the www-data user (Nginx's default user).
   • Incorrect Socket File Mode (listen.mode): The file permissions set on the socket file itself might be too restrictive, even if the group is correct.
   • Parent Directory Permissions: The directory where PHP-FPM creates the socket (e.g., /var/run/php/) might have permissions that prevent Nginx's user from listing its contents or accessing files within it.

2. Custom Unix Socket Permissions:

   • If you're using a custom Unix socket for Nginx to proxy to another service (e.g., another Nginx instance, a Node.js application, or a Python WSGI server), that service might be creating the socket with restrictive permissions, or the socket's parent directory might be inaccessible to Nginx.

3. Nginx Worker User Misconfiguration:

   • While less common, if the user directive in your nginx.conf has been changed from www-data to a user without appropriate privileges, Nginx itself could lack permissions to access various files, including sockets.

4. SELinux or AppArmor (Less Common on Default Ubuntu):

   • Although Ubuntu uses AppArmor by default, Nginx and PHP-FPM usually have pre-configured profiles allowing them to function. However, custom AppArmor profiles or SELinux (if manually installed/enabled) could restrict Nginx's access to specific file paths or socket types.

5. Temporary Directory Management by Systemd:

   • For services creating sockets in runtime directories (/run or /var/run), how Systemd manages these directories (e.g., RuntimeDirectory, RuntimeDirectoryMode) can influence permissions, especially for custom services.

Step-by-Step Resolution

Follow these steps to diagnose and resolve the "Permission denied" error for Nginx Unix sockets.

1. Identify the Affected Socket Path

First, determine which Unix socket Nginx is trying to connect to. This information is typically found in your Nginx site configuration files (e.g., /etc/nginx/sites-available/example.com.conf) or the main nginx.conf.

Look for directives like fastcgi_pass unix:/path/to/socket.sock; or proxy_pass http://unix:/path/to/socket.sock:;.

Example (for PHP-FPM):

# /etc/nginx/sites-available/example.com.conf
server {
    # ...
    location ~ .php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
    }
    # ...
}

In this common case, the socket is /var/run/php/php7.4-fpm.sock.

2. Verify Nginx Worker Process User and Group

Check what user and group Nginx worker processes are running as. By default on Ubuntu, this is www-data.

ps aux | grep nginx | grep worker

Expected Output:

www-data   12347  0.0  0.1  29268  5724 ?        S    14:31   0:00 nginx: worker process
www-data   12348  0.0  0.1  29268  5724 ?        S    14:31   0:00 nginx: worker process

If it's not www-data, note the user and group. If you intentionally changed this in nginx.conf using the user directive, ensure that user has appropriate permissions for the entire web server operation.

Changing the Nginx user from www-data without fully understanding the implications can introduce further permission issues or security vulnerabilities. It's generally recommended to adjust socket permissions rather than Nginx's user.

3. Inspect Socket File and Directory Permissions

Once you know the socket path, check the permissions of the socket file itself and its parent directory.

# Example for PHP-FPM socket
ls -ld /var/run/php/
ls -l /var/run/php/php7.4-fpm.sock

Example Output (Problematic):

$ ls -ld /var/run/php/
drwxr-xr-x 2 root root 60 Oct 26 14:30 /var/run/php/

$ ls -l /var/run/php/php7.4-fpm.sock
srw-rw---- 1 www-data www-data 0 Oct 26 14:30 /var/run/php/php7.4-fpm.sock

In the example above, the socket file php7.4-fpm.sock is owned by www-data:www-data with mode srw-rw---- (0660). This means the owner (www-data) and members of the group (www-data) can read/write. If the Nginx worker process runs as www-data, it should have access.

The problem often lies in the parent directory permissions (/var/run/php/ in this case). If /var/run/php/ is drwxr-xr-x 2 root root, then only root can modify it, but others can read/execute (traverse). This might be okay for Nginx to access a socket inside, but can be problematic if the socket itself has different ownership or group, or if the directory's group isn't www-data.

Crucially, if PHP-FPM creates the socket with owner:group as php-fpm:php-fpm and mode 0660, but Nginx runs as www-data:www-data, then Nginx won't have access.

4. Resolve PHP-FPM Socket Permissions (Most Common Fix)

The most robust solution for PHP-FPM related permission issues is to configure the PHP-FPM pool to create the socket with appropriate ownership and permissions for Nginx.

1. Edit PHP-FPM Pool Configuration: Open the relevant PHP-FPM pool configuration file. For PHP 7.4, this is typically:

   sudo nano /etc/php/7.4/fpm/pool.d/www.conf
   

2. Adjust listen.owner, listen.group, and listen.mode: Find the listen directive and the listen.owner, listen.group, and listen.mode settings.

   ; In /etc/php/7.4/fpm/pool.d/www.conf
   ; The address on which to accept FastCGI requests.
   ; Valid syntaxes are:
   ;   'ip.add.re.ss:port'    - Accept on a specific IP address and port.
   ;   'port'                 - Accept on all IPs and a specific port.
   ;   '/path/to/unix/socket' - Accept on a Unix domain socket.
   ; Note: This value is mandatory.
   listen = /var/run/php/php7.4-fpm.sock
   
   ; Set listen.owner and listen.group to the Nginx user and group
   listen.owner = www-data
   listen.group = www-data
   
   ; Set listen.mode to allow owner and group read/write access
   ; 0660 means owner (www-data) and group (www-data) have read/write, others no access.
   listen.mode = 0660
   

   Ensure listen.owner and listen.group match the user and group Nginx worker processes run as (usually www-data). The listen.mode 0660 is generally secure and sufficient.

3. Restart PHP-FPM and Nginx: Apply the changes by restarting the services:

   sudo systemctl restart php7.4-fpm
   sudo systemctl restart nginx
   

4. Verify: Check the Nginx and PHP-FPM status, and the socket permissions again:

   sudo systemctl status php7.4-fpm
   sudo systemctl status nginx
   ls -l /var/run/php/php7.4-fpm.sock
   

   The ls -l output should now show www-data www-data as owner and group, with srw-rw---- permissions.

5. Adjusting Parent Directory Permissions (Less Ideal, but Sometimes Necessary)

If the socket's parent directory itself (/var/run/php/ or a custom directory) has restrictive permissions preventing www-data from even traversing it, you might need to adjust them.

# Example: If /var/run/php/ is owned by root:root and Nginx needs write access to the directory itself
sudo chown root:www-data /var/run/php/
sudo chmod 775 /var/run/php/

Be very cautious when changing permissions on system directories like /var/run. Directly modifying these permissions is often a workaround rather than a fundamental fix. The preferred method is to configure the service that creates the socket (e.g., PHP-FPM) to do so with correct permissions in an appropriate directory. For /var/run/php/, PHP-FPM's systemd unit file often handles its creation with correct permissions.

If you have a custom service creating a socket, ensure its Systemd .service file uses RuntimeDirectory and RuntimeDirectoryMode to manage the directory properly:

# Example: myapp.service
[Service]
# ...
RuntimeDirectory=myapp
RuntimeDirectoryMode=0755
# This will create /run/myapp/ with 0755 owned by root:root by default,
# or by User/Group if specified in the service unit.
# For Nginx to access sockets in /run/myapp/, the socket itself must be accessible.

6. Check AppArmor/SELinux (Advanced)

While not a typical issue for default Ubuntu 20.04 Nginx/PHP-FPM setups, if you've customized security profiles or installed SELinux, these could be the cause.

• AppArmor: Check AppArmor status:

  sudo aa-status
  

  If nginx or php-fpm profiles are listed as enforced, examine /var/log/syslog or journalctl -xe for apparmor="DENIED" messages related to Nginx or PHP-FPM trying to access the socket path. Temporarily disabling AppArmor (if you suspect it) for testing is done via:

  sudo aa-complain /etc/apparmor.d/usr.sbin.nginx # Or the relevant profile
  sudo systemctl restart nginx
  

  Re-enable with aa-enforce. Consult AppArmor documentation for creating custom rules.

• SELinux: Check SELinux status:

  sestatus
  

  If SELinux is enforcing, check /var/log/audit/audit.log or use ausearch -c nginx --raw | audit2allow -M nginx_socket_fix to identify and potentially generate policies. Temporarily set SELinux to permissive for testing (not recommended for production): sudo setenforce 0.

7. Final Verification

After implementing a solution, always verify that your services are running correctly and your website is accessible.

sudo systemctl status nginx
sudo systemctl status php7.4-fpm # Or your custom service

Check the Nginx error logs again (sudo tail -f /var/log/nginx/error.log) for any new errors. Finally, navigate to your website in a browser to confirm functionality.

By systematically checking the socket path, Nginx worker user, and most importantly, the PHP-FPM pool configuration, you can effectively resolve the "Permission denied opening unix socket" error on your Ubuntu 20.04 LTS server.