NASLite Network Attached Storage

www.serverelements.com
Task-specific simplicity with low hardware requirements.
It is currently Fri Apr 19, 2024 7:22 am

All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 9 posts ] 
Author Message
PostPosted: Tue Apr 06, 2010 1:01 pm 
Offline

Joined: Thu Mar 09, 2006 2:55 pm
Posts: 26
Hello folks,

The reason I ask this question is this. I recently had two of my 500Gb drives fall completely offline. Because both dropped simultaneously, I initially thought it was a hardware issue but I was bale to identify that the hardware was working fine. Then I decided to try a disk repair via the SE console and found tons of errors - well, it looks like tons to me which is why I am asking.

Also, I wanted to also mention that just prior to the drives dropping offline, I was having huge issues in deleting files via both FTP and AFP connections - nothing would work and the drives failed to come back online so I decided to bounce the server and then, after reloading, the drives failed to mount - just showed FAILED in the startup screen.

I have attached the screen capture from the repair session.

Thanks in advance for any feedback.


Attachments:
driveRepairLog_2010-04-06.txt.zip [26.09 KiB]
Downloaded 981 times
Top
 Profile  
 
PostPosted: Tue Apr 06, 2010 7:22 pm 
Offline
Site Admin

Joined: Tue Jul 13, 2004 4:11 pm
Posts: 1771
Location: Server Elements
I don't know the details of your hardware, but errors on both drives could mean a number of things as follows:

1. You improperly powered off the machine without properly shutting down. An example of this is pulling the plug. That will result in errors especially if you had open files.

2. HARDWARE PROBLEMS - and that could mean a lot of things like excessive heat, power problems, RAM problems, cabling, connectors, etc.

In your post you eliminate hardware problems as the cause. How did you arrive to that conclusion? Not being able to manipulate the content on the drives via network sound like the hardware was not really doing it's job.

Any details you can provide and a syslog will be helpful here...


Top
 Profile  
 
PostPosted: Thu Aug 11, 2011 12:16 pm 
Offline

Joined: Thu Mar 09, 2006 2:55 pm
Posts: 26
Hello SE folks,

Dusting off this old thread and trying to get some more information...

Same issue but this time, all of the drives were fine - I have 2x 500GB and 2x 1.5T drives in a dedictaed enclosure. All was working great but I had to shutdown due to some remodeling going on in my SOHO. I performed a proper shutdown from the SE console and waited for the server to go lights out before powering down the drive enclosure so I feel pretty confident the system went down gracefully.

Anyway, as I brought her back up last weekend, the same two drives from my original post would not mount. Long story short, I had to perform the file system repair for them to come back properly and while I haven't yet had a chance to check all of the data, it seems fine thus far.

So - here is what I am wondering:

1) Is this EVER normal even after a proper shutdown?
2) If not, then aside from HW issues, what are the causes?
3) I did not see any reason in the startup log as to mounting failure - is this normal?
4) Is it possible for data loss to occur during this type of situation? I would think so but just double-checking.

Also, I have attached one of the repair logs for review if desired.

TIA


Attachments:
zeus_disk1_rebuild_2011-08-10.txt.zip [5.56 KiB]
Downloaded 944 times
Top
 Profile  
 
PostPosted: Fri Mar 02, 2012 9:35 pm 
Offline

Joined: Thu Mar 09, 2006 2:55 pm
Posts: 26
aleding wrote:
Hello SE folks,

Dusting off this old thread and trying to get some more information...

Same issue but this time, all of the drives were fine - I have 2x 500GB and 2x 1.5T drives in a dedictaed enclosure. All was working great but I had to shutdown due to some remodeling going on in my SOHO. I performed a proper shutdown from the SE console and waited for the server to go lights out before powering down the drive enclosure so I feel pretty confident the system went down gracefully.

Anyway, as I brought her back up last weekend, the same two drives from my original post would not mount. Long story short, I had to perform the file system repair for them to come back properly and while I haven't yet had a chance to check all of the data, it seems fine thus far.

So - here is what I am wondering:

1) Is this EVER normal even after a proper shutdown?
2) If not, then aside from HW issues, what are the causes?
3) I did not see any reason in the startup log as to mounting failure - is this normal?
4) Is it possible for data loss to occur during this type of situation? I would think so but just double-checking.

Also, I have attached one of the repair logs for review if desired.

TIA


Hello NL Team,

Any chance you could take a look and let me know your thoughts? I still have these drives in operation and they seem to be fine after the repair so I'm mainly looking for the answer to the questions posted so I can get a better idea of what proper vs. not proper with NL operation.

Thanks.


Top
 Profile  
 
PostPosted: Sat Mar 03, 2012 12:16 pm 
Offline
Site Admin

Joined: Tue Jul 13, 2004 4:11 pm
Posts: 1771
Location: Server Elements
aleding wrote:
1) Is this EVER normal even after a proper shutdown?
No, this is not NORMAL and should be addressed.

aleding wrote:
2) If not, then aside from HW issues, what are the causes?
Most likely formatted outside of NASLite. What's the history of the drives?

aleding wrote:
3) I did not see any reason in the startup log as to mounting failure - is this normal?
NASLite will not mount unclean filesystem. The filesystem is checked, fixed and verified prior to mounting. Can you post a syslog generated during a boot that has encountered errors?

aleding wrote:
4) Is it possible for data loss to occur during this type of situation? I would think so but just double-checking.
Abnormalities always carry increased probobility of failures, so yes, data loss is possible.

Hope the above helps.

Tony


Top
 Profile  
 
PostPosted: Sat Mar 03, 2012 12:49 pm 
Offline

Joined: Thu Mar 09, 2006 2:55 pm
Posts: 26
Hi tony,

Thanks for the information. Quick question, are previous startup logs retained or are the burned with each boot?


Top
 Profile  
 
PostPosted: Sat Mar 03, 2012 2:38 pm 
Offline
Site Admin

Joined: Tue Jul 13, 2004 4:11 pm
Posts: 1771
Location: Server Elements
aleding wrote:
Hi tony,

Thanks for the information. Quick question, are previous startup logs retained or are the burned with each boot?


Each boot is from bare-metal with no previously-saved states other than the config info. Note that the config does not contain any hardware-specific details other than the unlock.


Top
 Profile  
 
PostPosted: Tue Mar 06, 2012 1:30 pm 
Offline

Joined: Thu Mar 09, 2006 2:55 pm
Posts: 26
Hello Team,

So I observed some other anomalies that I beleive are evidence of the root cause causing the issues described at the start of this post. First, a little background...

The server has been running fine for about 5 days and has 4 total eSATA drives in an external enclosure. Each drive has a dedicated eSATA channel (e.g. vs. being on a single multiplexed channel) and make-up two mirrored sets. The mirror operation for the set in question is set to begin at 10pm nightly. While both drives still show as having a clean file-system after the mirror operation is complete, several errors related to the mirroring can be seen in the system message log. I have attached the whole log but also provided a snapshot of the some of the errors below:

Code:
Mar 4 22:15:10 [6] EXT3-fs error (device sdb1): ext3_valid_block_bitmap: Invalid block bitmap - block_group = 3218, block = 105447424
Mar 4 22:15:11 [6] EXT3-fs error (device sdb1): ext3_new_block: Allocating block in system zone - blocks from 105447424, length 1

---SKIP---

Mar 4 22:17:59 [6] EXT3-fs error (device sdb1): ext3_lookup: deleted inode referenced: 54730954
Mar 4 22:17:59 [6] EXT3-fs error (device sdb1): ext3_lookup: deleted inode referenced: 54730954
Mar 5 22:03:10 [6] EXT3-fs error (device sdb1): ext3_readdir: bad entry in directory #38381728: rec_len % 4 != 0 - offset=0, inode=3822110161, rec_len=63730, name_len=245
Mar 5 22:03:10 [6] EXT3-fs error (device sdb1): ext3_readdir: bad entry in directory #38381730: rec_len % 4 != 0 - offset=0, inode=3908564206, rec_len=54498, name_len=209
Mar 5 22:03:10 [6] EXT3-fs error (device sdb1): ext3_readdir: bad entry in directory #38381735: rec_len % 4 != 0 - offset=0, inode=4008634862, rec_len=60410, name_len=242
Mar 5 22:03:10 [6] EXT3-fs error (device sdb1): ext3_readdir: bad entry in directory #38381737: directory entry across blocks - offset=0, inode=4294967295, rec_len=65536, name_len=255
Mar 5 22:03:10 [6] EXT3-fs error (device sdb1): ext3_readdir: bad entry in directory #38381738: directory entry across blocks - offset=0, inode=3672302571, rec_len=61404, name_len=227


This is occurring with every mirror operation and the other mirrored set has nothing like this at all.

Does anyone have any idea what might be causing this? Do these errors indicate early signs of hardware failure?

Thanks to all those taking a look.


Attachments:
serverLog_zeus_2012-03-06-0855.txt.zip [10.68 KiB]
Downloaded 917 times
Top
 Profile  
 
PostPosted: Wed Mar 14, 2012 10:35 am 
Offline
Site Admin

Joined: Tue Jul 13, 2004 4:11 pm
Posts: 1771
Location: Server Elements
It may be a good idea to run any available OEM diagnostics on that drive. Do so using the same cable and controller to ensure that you are testing everything as it sits in your server.

Also, are there any mirror errors shown?

Tony.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 43 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group