Cant download 15,000 files via ssh linux [duplicate]

Posted on

Problem :

I have used wget, ncftp ftp etc, They all list around 5000 files at some sort of limit. I tryed using wget to get an exclusive file that didnt show up in the regular wget download and it worked. I know the files are there it just wont download all of them. help ASAP!!

Solution :

use rsync, it uses ssh and can handle a very large amount of files.

Have you tried downloading it recursively using wget --mirror?


If this is a FTP server only (or HTTP and FTP only) rsync won’t work: it’s a different protocol.

Try using secure copy (scp) with the recursive flag (-r).