New Disk-Mirroring Feature

Tony · **Posted:** Wed Dec 13, 2006 1:34 pm

Just a note to go with the new disk-mirroring feature:

If you are interested in using the feature to back up a drive hosted on a machine different than NASLite-2, all you have to do is set the source to export an anonymous rsync module named Disk-X as a valid NASLite-2 disk name. Permissions must also be accounted for, but as long as the path is valid, the mirroring should occur as necessary.

rscovell · **Joined:** Sun Sep 03, 2006 9:12 pm **Posts:** 7

Tony,

I have cygwin with rsync installed on my Windows XP box. I would like to use rsync to automatically backup my data drives from the Xp box to my NasLite server.

I have the following batch file on XP that I have been playing with, but I can't seem to get it to work.

Drive H is my XP drive
testrsync is a folder on drive H
192.168.1.250 is the NasLite server

c:cygwin\bin\rsync -vrt --delete "cygdrive/H/testrsync" 2.168.1.250:disk-0

error: can't find the path specified
any suggestions?

thanks
Bob C

Tony · **Posted:** Sat Dec 16, 2006 10:09 pm

The path is case sensitive:

192.168.1.250:Disk-0

t_indra · **Posted:** Tue Dec 19, 2006 6:26 am

Hi,
I am also trying to make use of the new rsync mirror feature, in a NAS to NAS setting.
But files on the target destination are not deleted as they should be.
I get an error:

Quote:

IO error encountered -- skipping file deletion

Here is the status page:

Code:

Mirror Details:

    Initiate Mirror Event:        Tue Dec 19 05:00:01 UTC 2006
Mirror Definition:            192.168.1.11/Disk-3/05
Source Host IP Address:       192.168.1.11
Source Volume Name:           Disk-3
Source Target:                Available
Initiate transport agent:     rsync  version 2.6.6  protocol version 29
IO error encountered -- skipping file deletion

Number of files: 4268
Number of files transferred: 0
Total file size: 16138636735 bytes
Total transferred file size: 0 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 68769
File list generation time: 0.564 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 279
Total bytes received: 69077

sent 279 bytes  received 69077 bytes  27742.40 bytes/sec
total size is 16138636735  speedup is 232692.73
Close transport agent:        Code (23)
Close Mirror Event:           Tue Dec 19 05:00:03 UTC 2006

NAS01 (IP 192.168.1.11) is the source and has rsync as RO enabled.
NAS02 (IP 192.168.1.12) is the target (and does not have rsync enabled).
This should be correct and permissible, as far as I understand the manual on p. 15/ top right)

To my understanding, I should be able to leave the source as RO, since I do not want any changes made there. The target is where files should be copied to, and deleted, if not present in source.

Any ideas why the error quoted above occured? (Rsync to/ from XP machines has been working as expected in the past)

And a couple more questions:
- The manual states, that the mirroring parameters can be manipulated via the admin util. Rsync as such has many more options available beyond source and time. How can additional options be set in the admin util? Or is this a future feature?
- What about slight time differences on the two servers: How much "Grace period" is assumed (until a file is considered "different" in time)?

Thanks in advance for shedding some light on this problem.

gaiden · **Posted:** Tue Dec 19, 2006 8:05 am

http://www.nabble.com/IO-error-encounte ... 67161.html

Anyone else having this problem?

googled it,

http://www.google.com/search?q=IO+error ... tartPage=1

Tony · **Posted:** Tue Dec 19, 2006 10:10 am

Hello Thorsten,

I was sure you’d be using this option with the large number of photos that you store on your NASLite servers. ;-)

The problem that you see is most likely caused by the “lost+found” folder on the destination drive. That should be handled in the next release (v2.04). To confirm the problem is as described above, simply delete the “lost+found” on the destination and observe the mirror log file after the next scheduled event completes.

Another thing that you can do is to take a look at the size and dates of the backed up files and make sure that things match the source.

I have been using the mirror feature for about 2 months now and am happy to say that it has not let me down, however I have to admit that I deleted the “lost+found” on both source and destination as a matter of (perhaps bad) habit.

You are correct that rsync supports a large array of features, however in the context of NASLite, those can be used only via a client. The mirroring in NASLite is built this way to allow a safe and stable disk backup strategy without the risk of damaging source data due to improper use of rsync. One can still do that via an open client, but then client capabilities are completely beyond our control. ;-)

Hope the above is helpful.

t_indra · **Posted:** Thu Dec 21, 2006 6:20 am

Hi Tony,
thanks for your reply.
I have tried the workaround of deleting the lost+found folder on the destination drive, but the error remains the same.

I have been using another excellent full featured synching solution thus far (Allsync), but would prefer using the NAS to NAS rsync to reduce system load on the Windows PCs.

Once this problem is dealt with with release 2.04: What about slight time differences on the two servers: How much "Grace period" is assumed (until a file is considered "different" in time)?
Since my servers don't seem to "hold" the time really well, that could pose a problem, if I am thinking right.

Synching the server time automatically with the NIST time server would be a great addition, BTW.

Tony · **Posted:** Thu Dec 21, 2006 10:22 am

Quote:

I have tried the workaround of deleting the lost+found folder on the destination drive, but the error remains the same.

How did you copy the source files onto the drive originally? Is it possible that there are files on the source with permissions not handled by NASLite? That can happen if you are using another OS to populate a drive natively prior to installing it into the NASLite machine.

---------
P.S.
What is currently on the destination drive in terms of content? Is the above true for the destination also?

Just trying to understand your environment and what the mirroring process has to deal with.

t_indra · **Posted:** Thu Dec 21, 2006 10:27 am

I have copied the files from NAS01 to NAS02, using Allsync, via a Windows XP machine. if I recall correctly, the drive was formatted under NASLite before being populated.

Tony · **Posted:** Thu Dec 21, 2006 10:29 am

Were any of the files in the destination possibly open or locked by a client at the time the mirror occured?

t_indra · **Posted:** Thu Dec 21, 2006 10:51 am

No, not at the destination.

t_indra · **Posted:** Thu Dec 21, 2006 11:02 am

Quote:

P.S.
What is currently on the destination drive in terms of content? Is the above true for the destination also?

Just trying to understand your environment and what the mirroring process has to deal with.

The destination drive is largely identical with the source drive, as I regularly back it up using Allsync. The drives contain primarily tifs and jpgs, mostly large (7-120 MB), about 32.000 files/ 400 folders.

The files on the source drive where also copied to a NASLite formatted drive.
If I remember correctly, I have used rsync as such ("detoured" via an XP machine) successfully in the past. (See also http://www.serverelements.com/phpBB2/viewtopic.php?t=1142&highlight=rsync+indra&sid=b685a6afd8a3629ed0c493074b51d9b5)

Tony · **Posted:** Thu Dec 21, 2006 12:06 pm

OK, if the files were written to both drives exclusively by NASLite via the network, then the problem is unlikely to be permission related.

The next item to look at is the amount of RAM. I've tested the mirror extensively on approximately 150,000 files (60G) using remote servers with 256M of RAM each. How much RAM is in the source machine and also the destination. How much free room on the drives?

I'm having a hard time with replicating the problem on my end. The lost+found folder is not the cause as i originally thought. The client rsync (mirror side) works fine and removes any artifacts that are not present in the source.

After the mirror error, do you see any pertinent content in the syslog?

If anyone else is experiencing this, please speak up.

t_indra · **Posted:** Thu Dec 21, 2006 4:07 pm

NAS01 (source) and NAS02 (target) both have 500 MB RAM.
Disk-1 has 7% in use, Disk-2: 75%, and Disk-3: 88%
The last few lines of the syslog read as follows:

Code:

Dec 18 07:45:49 user.warn kernel: VFS: Mounted root (ext2 filesystem). 
Dec 18 07:45:49 user.info kernel: Mounted devfs on /dev 
Dec 18 07:45:49 user.info kernel: Freeing unused kernel memory: 536k freed 
Dec 18 07:45:49 user.info kernel: e1000: eth0: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex 
Dec 18 07:45:49 user.info kernel: kjournald starting. Commit interval 5 seconds 
Dec 18 07:45:49 user.info kernel: EXT3 FS 2.4-0.9.19, 19 August 2002 on ide2(33,1), internal journal 
Dec 18 07:45:49 user.info kernel: EXT3-fs: mounted filesystem with ordered data mode. 
Dec 18 07:45:49 user.info kernel: kjournald starting. Commit interval 5 seconds 
Dec 18 07:45:49 user.info kernel: EXT3 FS 2.4-0.9.19, 19 August 2002 on ide2(33,65), internal journal 
Dec 18 07:45:49 user.info kernel: EXT3-fs: mounted filesystem with ordered data mode. 
Dec 18 07:45:49 user.info kernel: kjournald starting. Commit interval 5 seconds 
Dec 18 07:45:49 user.info kernel: EXT3 FS 2.4-0.9.19, 19 August 2002 on ide3(34,1), internal journal 
Dec 18 07:45:49 user.info kernel: EXT3-fs: mounted filesystem with ordered data mode. 
Dec 18 07:45:49 user.info kernel: kjournald starting. Commit interval 5 seconds 
Dec 18 07:45:49 user.info kernel: EXT3 FS 2.4-0.9.19, 19 August 2002 on ide3(34,65), internal journal 
Dec 18 07:45:49 user.info kernel: EXT3-fs: mounted filesystem with ordered data mode. 
Dec 18 07:46:00 auth.info login[3007]: root login on `pts/0' 
Dec 18 17:02:20 user.warn kernel: spurious 8259A interrupt: IRQ7. 
Dec 18 21:11:19 auth.info login[23923]: root login on `pts/0' 
Dec 19 09:35:20 user.warn kernel: spurious 8259A interrupt: IRQ15. 

Does this help any?

Tony · **Posted:** Thu Dec 21, 2006 5:51 pm

Do you have the same problem if you try and mirror a different drive?

I really want to get this resolved as I'm sure so do you ;-)

One other thing you can take a look at is memory usage. While the mirror event is active, take a look at the process stats on the system stats page. Keep an eye on the rsync memory usage and overall system load.

Next time you are on the site, check and see if Ralph or I are on the help line (the big blue button on the left hand side of the main site) and page us.

btw, the syslog looks fine at the bottom. The spurious interrupt is abnormal but harmless in most cases.

NASLite Network Attached Storage

New Disk-Mirroring Feature

Who is online