NASLite Network Attached Storage

I recently had a hardware error that caused me to worry a bit about how NL2 handles mirroring. What happened was that my NAS started acting up so I rebooted to see if that cleared things up. All seemed well until I noticed that one of the drives "disappeared". I have a NAS with 9 disks but only Disk-0 through Disk-7 was showing up. It turned out that the power supply had gone bad and Disk-4 had failed to power up. What worried me was that NL2 happily renumbered the disks so that what was supposed to be Disk-5 was Disk-4, etc. I have Disk-7 scheduled to mirror Disk-6 so some bad things could have happened if I did not notice the problem.

Is the mirroring setup in such a way so that NL2 would notice that things had changed or will it happily carry out the mirroring on the wrong drives?

NASLite-2 mirrors are initiated by the destination drive, so the source is not written to under any circumstances. The destination drive is not identified by it's notation (Disk-X) but by the mirror details saved on the drive. Every hour, NASLite-2 scans the drives for mirror details and acts accordingly. In other words, if you had Disk-7 set as a mirror destination and due to hardware problems it's now Disk-6, then Disk-6 will attempt to become a mirror of whatever the mirror details were set to copy from.

Using that approach, your source always remains intact while the mirror destination drive is absolute and is not affected by shift in notation order.

So...... if, say, Disk-4 is set up to mirror Disk-2 and Disk-2 fails, Disk-4 is is renamed Disk-3 and mirrors the contents of the old Disk-3 (now renamed as Disk-2) - you've lost the original Disk-2 and it's mirror on Disk-4 because that's now full of the original Disk-3.......

Sorry, but I am not sure how to interpret this. Can you tell me what would happen in the following scenario:

I have 9 drives and Disk-8 is set to mirror Disk-7.
Disk-6 is used for temporary storage that is deleted every few hours.
Disk-5 fails as I previously descibed so Disk-6, -7 and -8 are all "shifted" to become Disk-5, -6, and -7.
My "cleanup" routine runs and deletes everything on Disk-6. (Which used to be Disk-7, containing stuff I do not want to lose.)
The mirror job runs.

From what I can gather Disk-7 (old Disk-8) will now become a mirror of Disk-6 so everything will be deleted from it. Now if the original Disk-7 was damaged (due to the failing power supply) I've lost my data.

Please tell me I am wrong.

Consider the following:

1. If a disk fails and is unfit for data storage, it will not automatically drop out. A reboot is required for that to occur.
2. Further, it will issue an alarm to notify you that your attention is required, so prior to reboot, you'll be able to access the situation.
3. If the mirror event occurs while the source is bad, the mirror will likely fail with errors thus not overwriting the destination.
4. If the source drive fails to spin up or is removed from the chain for whatever reason, and the mirror event occurs, it will be mirroring what it is told and that is Disk-X. In that case, although the mirror is of other than the originally intended content, the original content should be intact and should not be lost from the source drive.

If a disk fails, you will know before rebooting. The mirror event will fail since the source can't be read and the mirror copy will be preserved. If however you deliberately take a disk out of the chain, or one drops out without your knowledge, then the mirror will be overwritten once the mirror event occurs with a different source. In that particular case however, the failure is largely an administrative one.

You are not wrong, you are most certainly right.

My advice would be that you have another NAS machine and use that for the temp files. Sounds like no matter how you set it up you are in for heartache if the temp file delete script runs and the drives have shifted.

Mike

This is all very disappointing. In all due respect to you Tony, you outline all the reasons why what I've experienced can't happen, and even if it did I would still not lose any data. True, the machine needed to be rebooted before all the disks were shifted, but there was no alarm. When the NAS started acting funny (I was getting write errors) I telnetted into it and did a reboot. There was no alarm at all, just a disappearing disk. The NAS machine runs headless in another room so I could not see all of the startup messages. It would not be uncommon to have a power suply failure cause a drive to give up the ghost so the possibility of losing the original data and the mirror is real.

I've tried to point out a big potential problem and I feel you just poo-pooed all over it. I'm just trying to help you make NASLite even better and more fool proof. I would think you could eliminate this problem by storing some of the details onf the mirror job in the MIRROR file and then if things did not "line up" you could abort the job. In my example if the mirror job "knew" that it was supposed to be Disk-8, and then suddenly found itself to be Disk-7 it would know something was wrong and send out an error beep.

Another solution would be to allow us to configure the shares with names we choose, but that is another can of worms that I'm sure you would quickly shoot down in the name of simplicity, reliability, and speed.

Mike - I do not actually use the drive for temp storage as I outlined, it is actually used for storing recorded TV shows (via SageTV). I used the temp storage scenario just to point out a big potential problem. The end results will be the same however, I'll end up with a mirror of a bunch of recorded TV shows and lose my real data. As a matter of fact when the disks "shifted" Sage started recording to "Disk-6" which was supposed to be "Disk-7" (which contains important data.)

Like I said, I'm just trying to help make the product better. Please do not take this the wrong way, but I've seen a lot of replies to posts along the lines of "well, that's crappy hardware you should be using something better" or "well, that's a silly thing to do and you should stop doing it." It's hard to get emotion and nuance across in forums so I will say again, please take this all as a satisfied user trying to make things even better.

Tom

Hay Tom,

Don't get me wrong here, I am not saying you are doing a stupid thing or anything like that. What I am saying is that there are very real limits with this software that you and I both hit, we either find a work around or live within the limits of it. Having been a very long time user of the products, I started with the free CD soon after it came out, I can attest to the fact that Tony and Ralph are not SH!tt!ing on your ideas or problems. What they have as a very clear idea of what the product is about and they stick to it. The product evolves in a very methodical way with features being added only as needed to improve the performance or feature set. If the addition will effect the performance of NL in a negative way they generally don't do it.


I am reaching the point were NL is a bit limited for my needs and will be taking an old machine and setting it up under Ubuntu as a file server. There are a number of offerings out there in the form of RAID management tools and software that allow very sophisticated file server back ends to be implemented fairly simply, for my needs and with my equipment this is the way for me. You may be at this point as well and not know it.

Mike

Mike - Agreed.

Yes, I see that - however, if each drive has a unique 32/48/64 bit disk-id (volume id?) could that not be incorporated very simply into a check before mirroring to ensure that if Disk-x is set to mirror Disk-y then Disk-y is actually Disk-y by checking the unique disk-id?

No, it should not be needed in the perfect works where everyone sits and watches as their server boots.

Yes, it might be needed in the situation where someone has maybe set up their Media Center PC to switch on the server at a pre-determined time using WOL so that recordings can be stored on it.

Well,

The point raised about shift in storage disk nomenclature and the way it affects the NASLite-2 mirror function is valid. I think that in the context of the mirror implementation, the significant item identifying the source is the port details. So, if a drive is connected as master on your primary IDE, then it can be targeted as "IDE, HOST-0, BUS-0, TARGET-0, LUN-0" for mirroring. We can hash those details in our next release, but the just of it is the existing mirror implementation needs to be improved by adding a check of port dependency prior to execution. If the port has changed, a mirror alarm will be issued.

That approach will preserve everything as is, including the existing mirror configuration and assignments, while accommodating the necessary checks in the management code, away from the user interface. So, things will remain as they are accept, if a disk drops, the mirror will fail.

Any comments are welcome...

Sounds like you have the fix for a problem that until now was not know of.

Mike

Essentially the problem will only occur if the source drive of the mirroring goes bad, user reboots the machine, bad drive doesn't mount (user leaves machine running till the next mirror) or user removes bad drive and doesn't reconfigure mirroring.

Tony and I discussed a solution today and it will be included in the next release.

Thank you!

You are welcome,

However I think you may have missed the point of my original post.

I think Mike was on target when he said:


NASLite as it stands is built to perform as a server. As such a number of assumptions were considered as part of the design criteria. One of those was that a server is not something you turn on and off all the time. In hind sight that assumption can be debated since many folks seem to do just that - turn their server on and off many times a day. As a practice that is very bad, however in reality, it appears to be the norm.

The considerations I listed were based on our original assumption that a server will be on, so if a drive fails NASLite will alert you by detecting SMART inconsistencies of the failed drive and issue an associated alarm. At that time you can take corrective action to resolve the problem. If a drive is bad at boot, it is as good as absent, so NASlite will not consider it as part of the chain. NASLite by the way has no clue what the hardware looked like before it was last shut down. Instead, it renders the hardware from scratch every time it is powered.

In your case, the drive did not come up fast enough and the BIOS ignored it, so did NASLite since for all practical purposes the drive wasn't there. The mirror on the other hand is an autonomous entity that simply makes itself a copy of a specified source - be it local or remote - once every 24 hours. We also assumed that there will be reasonable amount of time after boot and before the mirror event for one to find out a drive is missing from their chain. All I did in my original post is provide some considerations and agree with your assessment. It was not my intent to poo-poo all over your concern.

Now, with that off my chest, the original solution we presented will work locally but not remotely since there is no way for the mirror to acquire port details from a remote server. Some more thought on the subject is required, so Ralph and I will have to dig a bit deeper into this before we can define the final solution.

Perhaps the solution is not in modifying the mirror at all but marking the drives on boot. Don't know yet...

It's rarely as easy as it may seem.