Archive for August, 2010
Editing files across nine time zones
I often have to edit files and run programs on a server which is nine time zones away. We have a NoMachine server running there which is makes working with graphical user interface possible (standard X11 forwarding would be too slow…). But even NoMachine can be uncomfortably slow some times due to slow network. Working with an ssh login on a text based terminal however is quite convenient (emacs -nw is my friend here..).
Now recently, I tried mounting part of my home directory on the remote server via sshfs, so I could run more sophisticated development tools (e.g. netbeans) on my local computer while editing the source code and storing the compiled code on the server.
It turns out that with default settings, even sshfs is a bit slow. I tried googling around for a caching file system which I could run on top of the sshfs mounted directory. I came across a few solutions but they did not look too mature or supported to me.
At some point, I figured out that sshfs actually maintains a cache itself but the default parameters seem to be such that cache data is kept for 10 seconds only.
So I tried increasing all cache timeout parameters which I could find to two hours:
sshfs -o cache_timeout=7200 -o cache_stat_timeout=7200 -o cache_dir_timeout=7200 -o cache_link_timeout=7200 remote.host:/my/remote/dir /my/local/dir
Still, trying to do /bin/ls in the directory mounted from the remote host took six seconds even after several times when I would have thought that the information needed would be in the cache…
After some time, I ran /bin/ls with strace -r to see where it is taking most of its time:
0.000114 open("tls/x86_64/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
0.224241 open("tls/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
0.223974 open("x86_64/librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
0.224046 open("librt.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
(note that — if I’m understanding strace’s manpage correctly — the time printed at the beginning of each line is the time spent since the last call, i.e. the call in the first line in the above printout is actually the first taking a considerable amount of time)
So it seems that considerable time is lost while looking for librt.so in a subdirectory of the current directory (which is mounted from the remote server). As this file does not exist there (and neither does it exist in the cache), I would guess that sshfs tries to check on the server whether the file exists on the server or not and this takes some time.
Having seen this, I remembered that my LD_LIBRARY_PATH was not empty. No, I did not have the current directory (i.e. a dot) specified as part of the directories to search for but it ended in a colon which turns out to have the same effect. Having fixed this, ls produced its output instantly, without any noticeable delay.
So far, so good.
At some point, I wanted to copy a file from the (remote) server to the local PC. From my ssh login session I copied the file in question to the directory which I mounted on my local PC. Probably due to the long cache time out, this file did was not visible from my local PC… Creating a (empty) file in the same directory on the local PC seemed to cure this problem: the remote file suddenly appeared in the directory listing. I guess sshfs must have invalidated the cached information for this directory and updated the information from the server.
As I’m sure that nobody modifies my files on the remote server while I’m modifying them on the local machine, I can live with such extreme caching parameters and the limitation just mentioned above. In general though, it’s probably not recommended to use such settings.
Read Full Post | Make a Comment ( None so far )
Andre Holzner's Blog 