NASLite Network Attached Storage

www.serverelements.com
Task-specific simplicity with low hardware requirements.
It is currently Tue Jul 01, 2025 4:57 pm

All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 5 posts ] 
Author Message
PostPosted: Thu Aug 10, 2006 6:02 pm 
Offline

Joined: Tue Sep 14, 2004 11:18 am
Posts: 70
Location: Giessen, Germany
Hi,
in trying to optimize my backup strategies, I am now using rsync with the cygwin dll on a WinXP machine (as the client) and am rsyncing a mapped network drive/ disk from NAS01 to the corresponding disk on NAS02.
Ca. 30.000 files averaging 5-12 MB in 250 GB (x3) take an awful lot of time and processing power in the meantime, and are holding up one XP machine from doing more useful things. They are all networked using gigabit adapters.

So one thought was to dust off an unused Notebook with Win98, using the scheduler and the commandlines for rsync to relieve the main PCs and operating as a "backupserver" handling only the NAS-NAS rsync transfers. The notebook only has a 100Mbit LAN Card though, and only 256MB (?) of RAM (MMX Pentium 166Mhz).

Will such a setup (less RAM, 100Mbit LAN, slower processor) slow down the backup procedure, or can I still expect reasonable performance?
Or does anyone have any thoughts on how to best approach the given situation? :?
Any helpful hints are appreciated! TIA,


Top
 Profile  
 
 Post subject:
PostPosted: Thu Aug 10, 2006 8:56 pm 
Offline
Site Admin

Joined: Tue Jul 13, 2004 4:11 pm
Posts: 1771
Location: Server Elements
I'm not clear on your setup, but I assume you are using rsync to mirror between areas of two mapped drives. That is not the best approach. Rsync will be optimal as NAS1-server to NAS2-client, but NASLite is unable to act as a client, so next best thing is to go from rsync module on NAS1 to mapped disk from NAS2 where XP is the client. Again, not optimal, but good enough for local backups.

One thing to keep in mind is that with rsync you'll pay in CPU with what you gain in traffic.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Aug 11, 2006 4:29 am 
Offline

Joined: Tue Sep 14, 2004 11:18 am
Posts: 70
Location: Giessen, Germany
Hi Tony,
here are some more specifics on my current setup, together with the rsync command lines.

- 2 NASLite-2 USB servers, each running 4 disks. Named NAS01 (for "live/ production"-files = source) + NAS02 (as backup = target).
- one WinXP box acting as client in the rsync process
- Gigabit network
- \\NAS01\Disk-0 mapped as "I:\", with I:\IMG01 being the source folder
- \\Nas02\Disk-2\IMG01-BU being the target

The rsync commandline now looks as follows:
Code:
"C:\Program Files\rsync\rsync.exe" --progress -vv --archive --delete-after --sparse --stats --ignore-times --modify-window=1 --checksum "/cygdrive/I/IMG01/" "NAS02::Disk-2/IMG01-BU/" >>"Z:\22 PC LOGS&SETTINGS\rsync-logs\PHOTO\rsync-transfer.htm" 2>>"Z:\22 PC LOGS&SETTINGS\rsync-logs\PHOTO\rsync-error.htm"


To generate the file list alone for one drive (and I have three more drives to backup in a similar fashion) took over 7 hrs yesterday (in --dry-run mode) for about 39.000 image files with the setup described above. A bit long in my estimation.

So to avoid locking up the XP machine, my thought was to "clamp the old notebook in between", running rsync, to facilitate only the described NAS01 to NAS02 transfer. (See the notebook states in prev. message)

Does this make sense, or is there a more efficient way to use rsync or to do this backup in general?

Hope the added information helps to make sense of my scenario.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Aug 11, 2006 9:45 am 
Offline
Site Admin

Joined: Tue Jul 13, 2004 4:11 pm
Posts: 1771
Location: Server Elements
Looks good to me, however the 39,000 files is definitely a chunk. One thing to consider is that rsync requires a lot of resources on each machine it's running, especially when dealing with high file counts.

Regardless, I think the bottleneck is when parsing via the share. Ideally, the rsync transfer will occur from NAS to NAS so both filesystems are accessed locally by rsync. I don't know what is a clean solution for a NASLite rsync client, but I'll definitely have to give it some thought. It can be very helpful for sure.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Aug 11, 2006 4:46 pm 
Offline

Joined: Fri Feb 25, 2005 11:05 pm
Posts: 34
Location: Canada
Tony wrote:
Looks good to me, however the 39,000 files is definitely a chunk. One thing to consider is that rsync requires a lot of resources on each machine it's running, especially when dealing with high file counts.

Regardless, I think the bottleneck is when parsing via the share. Ideally, the rsync transfer will occur from NAS to NAS so both filesystems are accessed locally by rsync. I don't know what is a clean solution for a NASLite rsync client, but I'll definitely have to give it some thought. It can be very helpful for sure.


Tony are you suggesting you are considering adding an rsync client to NAS Lite ?

That would be fantastic !!!! I am working on using an off-site NAS Lite box to back up data on another NAS Lite server.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 5 posts ] 

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 14 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group