NAS Backup

I finished wiring up two buildings together with CAT6, a wireless bridge, and a NAS server for both PC and Mac. Now, I want to finally migrate my media files off of my main computer and have them be accessable to everyone on the network. However, before I do so I need to make sure my network storage is redundant and decently backed up.

I think the best way to go about backing things up without clogging my bandwidth doing internet backups would be to add another NAS. So I would have a primary NAS in building 1 while having a back up, read only NAS in building 2. The buildings are over 100 yards a part, so physical destruction of one house shouldn't affect the other (unless we have a tornado or something in which case I would have much bigger problems than data loss...)

The first NAS would create daily snap shots and replicate the drives on the secondary NAS. The secondary NAS would not be accessable by people on the network and it would be read-only.

Does this sound like the best way to go about it? I've also thought of doing USB over Ethernet to connect external hard drives to the first NAS and place them in the second building (physically away from the NAS). That way I wouldn't have to purchase a second NAS, but I'm unsure.

Lastly, with daily backups of the primary NAS, would it even be beneficial to run RAID 1? The most I would lose would be a day's work. I guess I could always upgrade later.

How would you guys approach this issue?

Layers, it's always about layers.

Do your RAID1 if that is your schema of choice. You might want to consider more frequent smaller rsync-style updates to the secondary NAS (hourly, etc.) to help keep network traffic from spiking.

You could also string several USB hdd's from the secondary NAS, allowing you to rotate them out to a 3rd geo-disperse location to make sure you have backups.

How far you take it depends on how dedicated you are willing to be (rotating out the USB drives to the off-site location etc.) in your data protection.

I personally have a NAS and a secondary NAS.
But I don't do off-site backups yet.

I started rebuilding my first NAS. I'm selling off my C2D set up and just ordered a Supermicro 1155 board with Kingston 2x4GB ECC RAM. I'll probably end up with 16GB once I get more drives online.

I haven't decided what CPU to use yet. I think a E3 Xeon would be total overkill for what I am doing. I'm thinking more along the lines of an i3-2120 or i3-2100. Microcenter sells the 2100 for 89.99, so that may be my best bet. Any opinions on this?

I'm thinking I will make the secondary NAS an off-the-shelf solution such as a Patriot Javelin or so. No need to go crazy if it will just be backing up. The USB HDD swap sounds like a good idea. Since my two sites are pretty far apart, I think I may be okay with just the two physically separated NAS's. However, I think it is always good to have an offline backup in case of intrusion or software problems.

Also, if I'm doing hourly backups would RAID1 even matter on the primary drive? The most I would lose would be an hour's work.

I'm upgrading my switches to 1GBbe. I was using some hand-me-downs haha.

Quote:

---

Originally Posted by Timman_24

I started rebuilding my first NAS. I'm selling off my C2D set up and just ordered a Supermicro 1155 board with Kingston 2x4GB ECC RAM. I'll probably end up with 16GB once I get more drives online.

I haven't decided what CPU to use yet. I think a E3 Xeon would be total overkill for what I am doing. I'm thinking more along the lines of an i3-2120 or i3-2100. Microcenter sells the 2100 for 89.99, so that may be my best bet. Any opinions on this?

I'm thinking I will make the secondary NAS an off-the-shelf solution such as a Patriot Javelin or so. No need to go crazy if it will just be backing up. The USB HDD swap sounds like a good idea. Since my two sites are pretty far apart, I think I may be okay with just the two physically separated NAS's. However, I think it is always good to have an offline backup in case of intrusion or software problems.

Also, if I'm doing hourly backups would RAID1 even matter on the primary drive? The most I would lose would be an hour's work.

I'm upgrading my switches to 1GBbe. I was using some hand-me-downs haha.

---

I started out with an E3-1230 (which is extremely awesome, but overkill). I ended up wanting that for my server, (DS, the one running minecraft), and instead put a P620 in the system. Runs fine with a 5-drive RAIDZ array under Nexenta. No slow downs or hiccups due to the slower CPU.

Quote:

---

Originally Posted by James

I started out with an E3-1230 (which is extremely awesome, but overkill). I ended up wanting that for my server, (DS, the one running minecraft), and instead put a P620 in the system. Runs fine with a 5-drive RAIDZ array under Nexenta. No slow downs or hiccups due to the slower CPU.

---

G620? I'll have to check the specs, but I'll need to make sure the Supermicro MB supports those. I know they support the i3-2100's and 2120's. I thought about the E3 line up, but they do seem very overkill for a file server unless I was compressing on the fly.

The G series would save me a lot of money, about 60 dollars. So the G620 is running the Minecraft server? I was thinking about a very low consumption processor with the T suffix, such as the i3-2120T or G630T.

I may just go with the G530 for 35 at microcenter. The good part about the 1155 socket is that I will have access to processing power that is fast enough for a file server for many years to come.

lol I have the following setup.

Supermicro X9SCM-F motherboard (Sandy Bridge E3 gen motherboard)
Pentium G620 2C/2T @2.6GHz
16GB (4x4GB) ECC unbuffered DDR3-1333 RAM

No, this is running my personal NAS. It's running Nexenta with ZFS for storage (which is all CPU based computation for the parity and checksumming of the filesystem).

The E3-1230 is running the minecraft server.

That's the exact same Supermicro board I bought. I got it for 77 dollars as an open box deal :) :). Nexenta costs a ton! Did you get a license from work or something?

http://www.nexentastor.org/projects/...mmunityEdition

The community edition is free for up to 18TB. How big a NAS are you building?

Quote:

---

Originally Posted by Vindir

http://www.nexentastor.org/projects/...mmunityEdition

The community edition is free for up to 18TB. How big a NAS are you building?

---

I just typed in Nexenta to google and the enterprise site came up. I'll have to look into it. How does it compare to freenas? I won't be using more than 18TB any time soon.

I love it. James talked me into it when I was building my new nas a few months ago and it has been solid and easy to deal with. Grafting the debian userland onto the solaris core was the biggest selling point to me aside from ZFS.

Haven't used freenas for anything major, but the difference you'll find will come down to native ZFS with nextena and open indiana versus the really old version they're using on freenas and linux distros. Last I checked it was something like version 33 on nexenta and version 15 on freenas. Freenas has more easily available "cool bits" like making it easy to use it as a bittorrent server and such. Comes down to your priorities.

Also worth looking at benchmarks. ZFS performance on anything other than a solaris base right now is pretty poor. http://www.zfsbuild.com/2010/09/10/f...fs-benchmarks/

Do you have a good guide you can point me towards? I'm totally new to the server world, I had to look up what 1U, 2U, etc meant is how ignorant I am :)

I just need to be able to serve files to Windows through CIFS and Mac through AFP, or something similar. I do not need a torrent box.

Honestly reading through that site (Zfsbuild.com) is a great general intro to an idea of how easy/hard ZFS is. They walk through basics like creating RAIDZ vdevs, creating pools, making backups, etc.

Nexenta CE is a wonderful middle ground. You can do everything via console if you prefer, or simply go through the GUI. Both of these options are extremely easy to do.

For example, in the GUI, there is a "shares" page, where all you have to do is check which protocols you want to share a given directory via, then click apply. No massive work trying to get things to work properly. Just a check box.

From the command line, if you wanted to share a folder (in this example, /data/smbshare) via CIFS you would type:

Code:

---

zfs set sharesmb=on data/smbshare

---

ZFS is almost disturbingly powerful and easy to use. Between that and several other factors, I really dig it. It's the first time I've gone with non-hardware based RAID and not worried about it.

Main notes both pros and cons that I think are worth knowing:
- ZFS likes RAM, the more the better
- ZFS checksums every bit of data on both write and read, as well as scheduled scrubs.
- ZFS is fairly CPU intensive as you scale the number/size of the vdevs, since all parity and checksumming is done via CPU.
- You can enable dedup and compression as well, but in general they will chew a ton of CPU. (I wouldn't recommend that G620 if you planned on doing this)
- ZFS makes snapshotting/backups extremely easy.


Read www.zfsbuild.com and a few others. Worth your time. :)

Thanks a lot James. I introduced myself to ZFS through freenas. I set up a box on an old C2D system and formatted a 320GB HDD using ZFS. I was able to serve AFB and CIFS, but I found freenas to be somewhat buggy. I ran into an issue where it wouldn't save properties without a restart due to an issue with the web GUI hanging. It also had issues where it would be recognized sometimes and then decide to drop out for some computers. Definitely did not instill confidence that I could rely on it, so I will definitely give nexenta a go.

I like the idea of having software RAID in an unbloated package. It makes upgrading hardware so much easier.

I'm still debating on whether to RAID1 my primary NAS or not. I'm thinking about doing a 500GB RAID1 for documents and important working files while doing the rest without RAID and rely on the daily/hourly replication to back those up.

I'm going to microcenter tomorrow, so I'll keep you guys updated on the build. I sold some old car parts I had in the garage, so I may splurge :D

Update, went to Microcenter today. It was my first time there and that place is extremely dangerous. I was like a kid in a candy store, I could spend thousands.

I ended up getting the i3-2100. I also decided to buy a Crucial M4 256GB SSD to replace my 2x1TB RAID1. I will move those drives (that are mainly used for media and steam games) to the server. I'll use the 256GB SSD exclusively for games and programs on my PC :).

I also bought a 2TB HDD. I've decided to do a 1TB RAID1 for very important files and a 2TB drive for media. Both will be backed up to USB attached external HDDs that will be rotated. I think that will be enough redundancy for now.

Eventually I would like to have a RAID1 of 2x500GB RE4's for important files, but we'll see. The rest of the parts come tomorrow.

Quote:

---

Originally Posted by Timman_24

Update, went to Microcenter today. It was my first time there and that place is extremely dangerous. I was like a kid in a candy store, I could spend thousands.

I ended up getting the i3-2100. I also decided to buy a Crucial M4 256GB SSD to replace my 2x1TB RAID1. I will move those drives (that are mainly used for media and steam games) to the server. I'll use the 256GB SSD exclusively for games and programs on my PC :).

I also bought a 2TB HDD. I've decided to do a 1TB RAID1 for very important files and a 2TB drive for media. Both will be backed up to USB attached external HDDs that will be rotated. I think that will be enough redundancy for now.

Eventually I would like to have a RAID1 of 2x500GB RE4's for important files, but we'll see. The rest of the parts come tomorrow.

---

You just couldn't resist buying a 256GB SSD could you? I have purchased a few of these Crucial M4 drives and they have been good thus far. I have them in various notebooks and those systems fly with those drives in them.

Quote:

---

Originally Posted by Nabby

You just couldn't resist buying a 256GB SSD could you? I have purchased a few of these Crucial M4 drives and they have been good thus far. I have them in various notebooks and those systems fly with those drives in them.

---

I know, I saw them for sale for 189 on the website and decided to just go for it when I got to the store. It actually comes out about the same as buying another 2x1TB HDDs for the server. Now I get to repurpose my PC drives as server drives. The point of the server was to offload storage from my main PC to a central location that everyone can access without having to run my main rig 24/7 (which is very power hungry.) I'll only keep games and programs on my computer now.

But yes, it turned out quite nice :D

I tried openindiana. I installed the OS and then tried to install napp-it, but on restart hadn't installed anything. Help showed that I may have been booting to a previous state or something, but eh. Now I'm installing nexentastor. I'm not liking how I have to install using a CD. I wish they had a bootable install USB option. The VMware looks painless, but I don't have that set up.

I got it up and running. I have a few questions for you guys though. With compression on, I'm only getting around 30MB/s on write. Does this sound about right? I'm writing movies, so they are large files, not a bunch of small files.

My setup is:
1x320GB HDD for system
2x1TB RAIDZ1

Compression on, dedup off

I wrote a smaller work load much faster, ~110MB/s. When ordered the server to write 60GB worth of movies, it went to 30MB/s, and I lost access to the GUI web interface. I'm guessing I am running into RAM issues. I think I may buy 2x40GB SSDs for a mirrored system drive and buy another 8GB of RAM for 16GB total.

Any suggestions?

Quote:

---

Originally Posted by Timman_24

I got it up and running. I have a few questions for you guys though. With compression on, I'm only getting around 30MB/s on write. Does this sound about right? I'm writing movies, so they are large files, not a bunch of small files.

My setup is:
1x320GB HDD for system
2x1TB RAIDZ1

Compression on, dedup off

I wrote a smaller work load much faster, ~110MB/s. When ordered the server to write 60GB worth of movies, it went to 30MB/s, and I lost access to the GUI web interface. I'm guessing I am running into RAM issues. I think I may buy 2x40GB SSDs for a mirrored system drive and buy another 8GB of RAM for 16GB total.

Any suggestions?

---

1. It's the compression that will be consuming all of the RAM and is probably your choke point (this depends on the type of compression used and is a guess.) If you turn it off (or make another share with it turned off in a different folder) does it go faster?

2. Extra RAM should help.

3. This is via CIFS I'm assuming? (SMB/CIFS share from Windows system). If so, you might not see much more than that. The chattiness of the old SMB1 protocol really limits transfer speeds. The newer SMB2 helps a lot, but it is still Windows only (with Samba4 alpha being the unstable exception).

I've got it working with compression at ~100MB/s with large files now.

EDIT: I got that stuff worked out

I'm now migrating a bunch of files over. I tested the redundancy by pulling a drive out. I reattached the drive and it rebuilt the pool fine. I was still able to access files and change them with the drive down, so that was nice. I wish the GUI let you know with big flashing lights though. The fault is kind of buried in the UI.

All I have to do now is set a backup to an external drive and I'll be all set. Its nice having all my files off of my computer. The permissions were strange, but I think I got it mostly figured out. It is funny how my MacBook Pro makes it easier than Windows 7 to set up a CIFS share...

Did you add more ram or find something else that was bottle-necking your speed?

Quote:

---

Originally Posted by Vindir

Did you add more ram or find something else that was bottle-necking your speed?

---

Well... the movies I was transferring were on a USB 2.0 device... :o I'm not totally 1Gbe on my network. I've got that wifi bridge, so computers from my house only see ~7MB/s which is saturating the wifi link. From computers connected via CAT6 through my Gbe switch, they see 100-110MB/s.

I haven't tested simultaneous read/writes from multiple computers yet. I'm working to figure out how to set up USB backups now. I was also wondering if it was possible to rsync with amazon s3. I'd love to do that with my critical documents on a weekly basis.

Check out the root shell. http://greg.porter.name/wordpress/?p=646

Could set the rsync up as a cron job there. Might need to figure out UUIDs and add some bash scripting to handle failures when the drive isn't hooked up.