rsync: File name too long (91)

20 replies [Last post]
tk
Online
Joined: 01.05.2008

Hi All,
I'am using cwRsync 2.0.10 and have encountered rsync:File name too long (91) on really long file names. Investigation shows that cygwin have MAXPATHLEN much shorter than that of linux, it can be simple patch for cygwin but I don't want to recompile cwRsync with bigger MAXPATHLEN.
Maybe some one know a way to deal with it without recompilation? Or maybe the reason why MAXPATHLEN in cygwin so short?

tk
Online
Joined: 01.05.2008

THe limitation is about 260 chars and comes from the cygwin layer.

tk
Online
Joined: 01.05.2008

yes i know its cygwin limitation, question is can you change it when copmpiling next version of cwRsync or that limitation have to be there?

tk
Online
Joined: 01.05.2008

Cwrsync uses standard cygwin. As long as cygwin has this limitation, cwrsync has also the same.

teddy42
Offline
Joined: 01.05.2008

Hi,

I have the same problème... But it's very important in my project to find a solution for those files... Embarassed What are the way to solve this limitation ? I need to rebuild a new version of the cygwin dll, there is something else to do ?

Thanks in advance for your help/advice ! Smile

Teddy

patrick1069 (not verified)

I have tried upgrading to the newest cygwin (1.5.24-2) to no avail.  Has anyone devised a workaround for this problem?  I am also stuck on this as the windows machine has 1000s of very long paths and file names that rsync will give "rsync: readlink [long path/file name here] (in test) failed: File name too long (91)"

wsherman (not verified)

For cygwin to use longer path names, the work is partially done.  Cygwin needs to use the unicode win api calls.  See here for a patched version does that:

  http://www.okisoft.co.jp/esc/utf8-cygwin/

Then, a couple of other things need to be changed.  Here is a quote from an email conversation with the author of the cygwin-utf8 patches:

(12/7/2006 7:11 PM)
 Wayne wrote:

    Thanks for your work on the UTF-8 patches.  I am using your cygwin1.dll for rsync backups.  I hoped it would bring the advantage of longer paths, but I am getting ENAMETOOLONG - "File name too long (91)" errors when using Rsync.  Using the wide (unicode) win32 api calls brings another advantage besides supporting special characters.  Many of the unicode (wide) functions support a 32000 character path length instead of 260 characters.

Note what the following page says:
  http://msdn2.microsoft.com/en-gb/library/aa365247.aspx

<BEGIN QUOTE>

Maximum Path Length

In the Windows API, the maximum length for a path is MAX_PATH, which is defined as 260 characters. A path is structured in the following order: drive letter, colon, backslash, components separated by backslashes, and a null-terminating character, for example, the maximum path on the D drive is D:\<256 chars>NUL.

The Unicode versions of several functions permit a maximum path length of approximately 32,000 characters composed of components up to 255 characters in length. To specify that kind of path, use the "\\?\" prefix.

Note  The maximum path of 32,000 characters is approximate, because the "\\?\" prefix can be expanded to a longer string, and the expansion applies to the total length.

For example, "\\?\D:\<path>". To specify such a UNC path, use the "\\?\UNC\" prefix. For example, "\\?\UNC\<server>\<share>". These prefixes are not used as part of the path itself. They indicate that the path should be passed to the system with minimal modification, which means that you cannot use forward slashes to represent path separators, or a period to represent the current directory. Also, you cannot use the "\\?\" prefix with a relative path. Relative paths are limited to MAX_PATH characters.

When using the API to create a directory, the specified path cannot be so long that you cannot not append an 8.3 file name.

The shell and the file system may have different requirements. It is possible to create a path with the API that the shell UI cannot handle.

<END QUOTE>

It would be nice if you include in your patch the advantage of longer path names.  Items that would have to be considered:

    1) Where does cygwin use the windows MAX_PATH constant
    2) How is CYG_MAX_PATH used?  Can it be changed to something longer?
          (rsync uses #define MAXPATHLEN 1024)
    3) If there is the possibility of handling relative paths, these must be transformed to absolute.
    4) prepend "\\?\" to the path names before you call the wide api calls (such as before calling CreateFileW)

What do you think of this idea?

It is very interesting.

    I am sorry for my late response.  As for handling path
names in Cygwin, you should look into "class path_conv" in
winsup/cygwin/path.h first.  Its private member "char path[CYG_MAX_PATH]"
holds the path name to be processed.  It represents both a posix (Unix)
path name and a Windows path name, which is why it is difficult to
incorporate Unicode into the middle of cygwin1 and I have taken a wrapper
approach.

    Just redefining CYG_MAX_PATH and modifying the wrapper functions
may be sufficient to handle long path names.  But I do not have much time
to try it out for now.  I am hoping you will try it for yourself.  The
patch at http://www.okisoft.co.jp/esc/utf8-cygwin/ is open for everyone.
To compile it:

$ tar xf cygwin-1.5.22-1-src.tr.bz2
$ cd cygwin-1.5.22-1/winsup
$ bzcat ~/winsup-utf8-patch-08-09.diff.bz2 | patch -p0
$ cd ..
$ ./configure
$ make

You will get new-cygwin1.dll in i686-pc-cygwin/winsup/cygwin.  This is
explained in Japanese at
http://www.okisoft.co.jp/esc/utf8-cygwin/download.html
and I hope you read the command lines there.

Best Regards,

Wayne 

totologie (not verified)

Hi *all

I'm French, so my english is not very good :-(

Thx for this very good work...

I have the same problem but I don't understant how to resolv it.

 I use cwRsync_2_1_0 on WinXp pro.

I tried to replace cygwin1.dll by http://www.okisoft.co.jp/esc/utf8-cygwin/ version but no result :(.

 does it exist a full win32 install which take into account this patch ?

Thx for your help

Regards 

 

teddy42
Offline
Joined: 01.05.2008

Hi,

I use cwRsync every day and I have a lot of "File name too long" and "File has vanished" errors. I need to find a solution, but I don't have suifisents skills to solve the problem... So, I propose to pay something to the person he solve this problem ! It's not a joke !

Who has a solution ?

 

Ps: sorry for my english...

Teddy

infocom
Offline
Joined: 01.05.2008

Hi. Just wondering before I spend time on this solution if it works? I think this is a serious issue that
needs an easy solution because it renders rsync useless if you have lots of long filenames because your backup wont be right.

My biggest issue is I am using rsync to backup my CPanel folders and the mail files are too long. Its important I make a backup of this mail. I can download these files from my Linux to Windows server using FTP and the filenames so not an issue, it must be with rsync.

tk
Online
Joined: 01.05.2008

Unfortunately,  this limitation is cygwin specific. AFAIK, the coming version (1.7.x) will address this issue. No idea when it will happen.