Troubleshooting ‘rsync: some files could not be transferred permissions’ on Debian 12 Bookworm

Resolve common rsync permission denied errors on Debian 12. This guide provides step-by-step solutions for local, remote, and SSH-related permission issues.

When performing file synchronization or backups using rsync on Debian 12 Bookworm, encountering "some files could not be transferred" errors, often accompanied by "Permission denied" messages, is a common operational hurdle. These errors typically indicate that the user initiating the rsync command, or the user it connects as on the remote system, lacks the necessary read or write privileges for specific files or directories. This guide provides a comprehensive, highly technical approach to diagnose and resolve these intricate permission-related rsync failures, ensuring your data transfers are robust and reliable.

Symptom & Error Signature

The most prominent symptom is an rsync command that completes with a non-zero exit code, indicating partial failure, and outputs specific "Permission denied" messages in its standard error stream. You might see a summary line similar to rsync error: some files/attrs were not transferred (code 23) or rsync error: some files could not be transferred (code 23) at the end of the output.

Here are typical error outputs you might encounter:

$ rsync -avzh /source/path/ user@remote:/destination/path/
sending incremental file list
rsync: [sender] chdir "/source/path/restricted_dir" failed: Permission denied (13)
rsync: [generator] failed to set times on "/source/path/.": Operation not permitted (1)
rsync: [sender] recv_file_list: failed to open base file . for ._some_file.ext: Permission denied (13)
rsync error: some files/attrs were not transferred (code 23) at main.c(1828) [sender=3.2.7]
rsync error: received SIGUSR1 (code 19) at main.c(1243) [receiver=3.2.7]

Or, if the issue is on the remote (destination) side:

$ rsync -avzh /source/path/ user@remote:/destination/path/
sending incremental file list
some_file.txt
rsync: [receiver] mkstemp "/destination/path/.some_file.txt.XXXXXX" failed: Permission denied (13)
rsync error: some files could not be transferred (code 23) at main.c(1828) [sender=3.2.7]

The key indicators are "Permission denied (13)" and "rsync error: some files/attrs were not transferred (code 23)". Code 23 specifically means "Partial transfer due to error".

Root Cause Analysis

Permission-related rsync failures are almost universally due to one or more of the following underlying issues:

1. Insufficient Local Source Permissions: The user running the rsync command on the source machine does not have read access to the files or directories being transferred. This can happen if files are owned by root or another user, and the rsync user is not part of the correct groups or lacks global read permissions.
2. Insufficient Remote Destination Permissions: The user rsync connects as on the remote machine does not have write access to the target directory or the necessary permissions to create temporary files (which rsync often uses) within that directory. This is the most frequent cause for "Permission denied" errors during actual file transfer on the receiver side.
3. Incorrect SSH User or Authentication: The rsync command might be attempting to connect to the remote host using an SSH key or user that does not have the expected permissions on the remote system.
4. Preserving Permissions/Ownership (-a flag): When using rsync -a (archive mode), rsync attempts to preserve permissions, ownership, and timestamps. If the remote user is not root, it often lacks the privilege to set ownership (chown) on received files, leading to errors. While this doesn't always stop the transfer, it can result in warnings or failures if strict preservation is enforced.
5. Immutable File Attributes (chattr +i): Less common but critical, files or directories on either the source or destination might have the immutable attribute set (chattr +i). This prevents any modification, deletion, or renaming, even by root.
6. Access Control Lists (ACLs): Modern Linux filesystems often support ACLs, which can impose finer-grained permissions that override or supplement traditional chmod permissions. If ACLs are in place, they might restrict access even if ls -l shows adequate permissions.
7. SELinux or AppArmor Policies: On some systems, Mandatory Access Control (MAC) systems like SELinux (rare on default Debian but possible) or AppArmor (common on Debian) can restrict what processes can do, regardless of standard POSIX permissions. A policy might prevent rsync from writing to a specific directory, even if the user has rwx permissions.

Step-by-Step Resolution

Follow these steps to systematically diagnose and resolve your rsync permission issues.

1. Verify Local Source Permissions

Ensure the user executing rsync on the source machine has read access to all files and execute access to all directories within the source path.

1. Identify the executing user:

   whoami
   

   (e.g., sysadmin)

2. Check directory permissions: Recursively list permissions for the source path:

   name_of_user=$(whoami)
   sudo -u "$name_of_user" ls -ld /source/path
   sudo -u "$name_of_user" find /source/path -type d -print0 | xargs -0 -I {} bash -c 'echo -n "{}: "; sudo -u "$name_of_user" ls -ld "{}"'
   

   Ensure the executing user or their group has r-x (read and execute) on directories.

3. Check file permissions:

   sudo -u "$name_of_user" find /source/path -type f -print0 | xargs -0 -I {} bash -c 'echo -n "{}: "; sudo -u "$name_of_user" ls -l "{}"'
   

   Ensure the executing user or their group has r-- (read) on files.

   If permissions are too restrictive, adjust them temporarily for testing:

   chmod -R u+rX,g+rX /source/path
   

   This grants read/execute to the user and group recursively. Avoid 777 unless absolutely necessary for testing, and revert immediately.

2. Verify Remote Destination Permissions and Ownership

This is often the most critical area. The user rsync connects as on the remote host must have write access to the destination directory.

1. Determine the remote user: If you use rsync user@remote:, user is the remote user. If omitted, it defaults to your local username.

2. Log in as the remote user:

   ssh user@remote
   

3. Check destination directory permissions: Once logged in, navigate to the parent directory of your target destination and check permissions:

   ls -ld /destination/path
   

   The user you logged in as (or their group) must have rwx (read, write, execute) permissions on /destination/path.

   rsync typically creates temporary files (e.g., .filename.XXXXXX) in the destination directory before renaming them. Therefore, the remote user needs not just write permissions, but also execute permissions on the destination directory, as well as the ability to create and delete files within it.

4. Adjust remote permissions (if necessary):

   # As root or sudo user on the remote host
   sudo chown -R user:group /destination/path
   sudo chmod -R u+rwx,g+rwx /destination/path
   

   Replace user and group with the appropriate remote user and group that rsync is connecting as.

   Be very careful with chmod -R on critical directories. Always test with a small, non-production directory first.

3. Ensure Correct SSH User and Key

Confirm rsync is using the intended SSH user and that the key-based authentication is working correctly for that user on the remote.

1. Test SSH connection:

   ssh -v user@remote
   

   The -v flag provides verbose output, helping diagnose SSH authentication issues.

2. Specify SSH user explicitly: Always specify the remote user in your rsync command:

   rsync -avzh -e "ssh" /source/path/ remote_user@remote_host:/destination/path/
   

4. Adjust rsync Options for Permission Handling

If rsync -a (archive mode) is causing problems due to ownership preservation, you might need to relax these requirements.

1. Disable owner/group preservation: If the remote user is not root, they cannot change file ownership. Use --no-owner and --no-group to skip setting these attributes.

   rsync -avz --no-owner --no-group /source/path/ user@remote:/destination/path/
   

   This is often the solution when remote "Operation not permitted" errors occur specifically for chown or chgrp.

2. Set owner/group explicitly (if applicable): If you want files to be owned by a specific user/group on the remote, regardless of the source, use --chown.

   rsync -avz --chown=remote_user:remote_group /source/path/ user@remote:/destination/path/
   

   --chown requires the remote rsync process to have sufficient privileges (e.g., be running as root or a user with chown permissions over the files/directories). This usually implies using sudo on the remote for rsync (see next step).

5. Using sudo on the Remote End for rsync

For situations where the remote user explicitly needs root privileges to write to certain locations or preserve ownership/permissions (e.g., rsyncing to /var/www or /opt), you can configure rsync to use sudo on the remote side.

1. Ensure sudo is configured for the remote user: The remote user must be in the sudo group or have specific sudoers entries allowing them to run rsync as root without a password. On Debian, add the user to the sudo group:

   # On the remote host, as root
   usermod -aG sudo user
   

   Then, confirm it works: ssh user@remote 'sudo rsync -V' should work without prompting for a password. If it prompts, check /etc/sudoers or /etc/sudoers.d/.

2. Modify the rsync command: Use the --rsync-path option to prepend sudo to the remote rsync command.

   rsync -avz --rsync-path="sudo rsync" /source/path/ user@remote:/destination/path/
   

   This tells the local rsync to execute sudo rsync on the remote side instead of just rsync.

   Using sudo for rsync on the remote side grants significant privileges. Ensure your SSH access is secured (key-based authentication, strong passphrases, no password authentication) and restrict sudo access as much as possible in /etc/sudoers if security is paramount. For example, limit sudo rsync to specific paths.

6. Check for Access Control Lists (ACLs)

ACLs can override standard chmod permissions. If you suspect ACLs are interfering, check them.

1. Install ACL utilities (if not present):

   # On source/remote, if not already installed
   sudo apt update
   sudo apt install acl
   

2. Check ACLs on relevant directories/files:

   getfacl /source/path
   getfacl /destination/path
   

   Look for specific user: or group: entries that might deny access even if ls -l shows open permissions.

3. Remove problematic ACLs (if found):

   # Remove a specific ACL entry
   sudo setfacl -x u:user /path/to/dir
   # Remove all default ACLs
   sudo setfacl -b /path/to/dir
   

   Removing ACLs can change access for other users/processes. Proceed with caution and understand the implications.

7. Check for Immutable File Attributes (chattr +i)

The immutable attribute can prevent even root from modifying or deleting a file.

1. Check for immutable attributes:

   lsattr -d /source/path
   lsattr -R /source/path
   lsattr -d /destination/path
   lsattr -R /destination/path
   

   Look for the i flag in the output (e.g., ----i--------e-- /path/to/file).

2. Remove immutable attributes (if present):

   sudo chattr -i /path/to/file_or_dir
   

   Immutable attributes are a strong security feature. Understand why they were set before removing them.

8. (Advanced) Investigate SELinux or AppArmor

While less common to be the primary cause on a default Debian 12 setup for simple rsync operations, these MAC systems can impose restrictions.

1. Check AppArmor status:

   sudo aa status
   

   If AppArmor is enforcing profiles for processes involved (e.g., rsync if a custom profile exists, or other services interacting with the files), you might see denials in the kernel logs.

2. Check system logs for denials:

   sudo journalctl -xe | grep "AppArmor"
   sudo dmesg | grep "denied"
   

   Look for "denied" messages related to rsync or the target path.

   Troubleshooting AppArmor or SELinux is highly environment-specific and requires deep knowledge of policy writing. As a temporary diagnostic, you could put AppArmor into complain mode for a profile (sudo aa-complain /path/to/profile), but this is generally not recommended for production without careful consideration. Usually, if AppArmor is the cause, it means rsync is interacting with a directory managed by another service (e.g., web server document root with an Nginx AppArmor profile). In such cases, modifying the profile to allow rsync write access might be necessary.

By systematically working through these steps, starting with the most common causes (local/remote user permissions) and moving to more advanced considerations (ACLs, immutable attributes, MAC systems), you will successfully identify and resolve rsync permission errors on your Debian 12 Bookworm servers.