What NAS boxes run on open source software that can be compiled by the user? - Page 1
Forum section: electronics / general

author: cedric!
author_rank: Contributor
timestamp: November 27, 2020, 06:56:46 pm
content: Hi All,I'm looking to buy a NAS, but I would like to (be able to) compile and package the operation system of the NAS myself. Replacing the closed source OS from the vendor with Debian of freeNAS is also a valid option.The point is, I don't want to be dependent on a vendor for updates, and I also like to be able to use the NAS for something else entirely when I'm bored using it as a NAS.Is this possible with a QNAP or a Synology NAS?

author: fordem
author_rank: Regular Contributor
timestamp: November 29, 2020, 07:05:42 pm
content: I think you need to recognize that whether or not what you wish to achieve can be done, is going to depend entirely onyourability to do it.A NAS is essentially a computer with one or more network interfaces and one or more disks attached to it - most of the commercially available NAS boxes run a linux based OS and provide code under various GPL licenses, but, that last part is not really relevant to this discussion because your stated intent is that you want to compile & package that yourself.There is no reason to limit your choice to commercial available NAS boxes like the QNAP or Synology product, but, if you choose to do so, you'll need to tailor your OS to whatever hardware they put in the model you choose, the question is no longer what they allow you to do, it's now do YOU have the skills to make what you want happen on that piece of hardware.For what it's worth - I'm now on to my third NAS - NAS #2 was a D-Link product that was thoroughly hacked, not to the point where I compiled & packed my own OS, but I certainly had it doing tasks that D-Link never intended it to do. When NAS #2 died, it was replaced with #3, a Netgear ReadyNAS which allows the user to run apps of his/her own creation - I have not yet gotten the itch to hack the Netgear.

author: bd139
author_rank: Super Contributor
timestamp: November 29, 2020, 08:12:38 pm
content: Sounds like you need a PC with some storage attached and some NAS software rather than a NAS that is open source to run on existing hardware.

author: ZigmundRat
author_rank: Regular Contributor
timestamp: November 30, 2020, 01:20:43 am
content: It might not be so easy to put open software on closed hardware. That is, using QNap or Synology hardware with Debian, etc. Their hardware also does not typically have a lot of extra resources, although adequate for the job they are designed to do.If I wanted a multipurpose system similar to what you describe, I would look into OpenMediaVault as it’s based on Debian, uses standard apt package management, has a good web UI to handle the NAS complexity, etc. It frees you up to use hardware more suitable for desktop, and with the proper case, plenty of drive slots.As a Synology fan I went the XPenology route, using an 8 bay case from U-Nas.com, a mini ITX motherboard I already had (i5 3.4GHz, 16 gb ram), and a Dell/LSI Logic SAS/SATA RAID card. This system would just as easily run FreeNAS, OpenMediaVault, RockStor, or even plain Debian/Ubuntu. But I wanted the packaged NAS software, and it closely matches the DS1813 I already have.

author: PKTKS
author_rank: Super Contributor
timestamp: December 07, 2020, 01:59:25 pm
content: Quote from: cedric! on November 27, 2020, 06:56:46 pmYOU DO NOT NEED ANY SORT OF BOTH...Neither buy some sort of "GUI" applet thing..or compile it yourself..What those folks of NAS products are doing is to wrapa fancy "configure" GUI thing over a linux distro..And there are already more than 100 to choose.I have built my NAS entirely from my scratch based systemand the easiest way to go is** SYSTEMD FREE with BUSYBOX core structure **small light and very scalable..can run entirely into a safe memory mapped imageFYI mine runs fine for more than decade...You just need to decide what features to bundle.IMHO?- strong minimalist kernel- systemV init scripts to configure a strong firewall policyand proper REDUNDANT attached bonded interfaces.- redundant Ethernet ports balanced traffic and fault tolerant(impossible to achieve with single port)- BONUS if your router can cope with that and VLAN altogether.- EXT4 is the most rock solid and fast responsive FS for nowthat includes RAID and LVM- alternatives tend to present issues along time. EXT4 does not.pretty much that simplePaul

author: EasyGoing1
author_rank: Regular Contributor
timestamp: February 25, 2021, 12:18:13 pm
content: Quote from: bd139 on November 29, 2020, 08:12:38 pmI'm not so sure about that ... I mean go to newegg and find me anything remotely as cool or capable as the hardware that Synology sells in a NAS style box. Though I'm a huge fan of DSM myself, I could see the appeal of using Synology with an open-source NAS OS. Some of us like grape, others strawberry ... neither is incorrect.

author: EasyGoing1
author_rank: Regular Contributor
timestamp: February 25, 2021, 12:26:22 pm
content: Quote from: PKTKS on December 07, 2020, 01:59:25 pmPLEASE tell me that this is not a config that you actually utilize and benefit from in a home environment. If it is, I'd be really curious to know the specifics of your fault-tolerant ethernet setup and stories about when you actually needed that. Implementing VLANs, of course, makes sense for the whole fault-tolerant thing but it just screams overkill for a home network ... unless it's just something you do for fun in which case... by all means have at it! Still would love to know the details of that redundant link though.Mike

author: PKTKS
author_rank: Super Contributor
timestamp: February 25, 2021, 01:41:16 pm
content: Quote from: EasyGoing1 on February 25, 2021, 12:26:22 pmActually this **IS** the config I use in all my BENCH setup.I have all workstations with 3 NICS BONDED with a compositeinterface so the first layer is traffic balance and the second layer isfault tolerant..If ONE interface on the MASTER fails the second still works..if THE CABLE FAIL the FAULT SLAVE interface takes place...Enough to serve a rather half dozen stations servingNAS + ZoneMind... local integrated FTP/HTTP and stuff...bonus that I still play some games w/my 5y old nephew.. (6 next week)in our local game server... (one machine is dedicated to that stuff)Cabling is managed by a bunch of switches in which the mastergateway provides VLAN ...Detail that matters is that your switches should be able to copeMode 802.3ad **NOT** 802.3abotherwise you will *NOT* be able to do LINK AGGREGATIONIt is not for first time pilots... as it takes a rather experience tosetup such BONDED INTERFACES with link aggregation andrequire some skills on configuring your switches ... router andgateways..That setup also holds my BENCH instruments w/a bunch ofhttp and usb stuff..I rather prefer they could be attached to ETH but mostly they are USBUSB sucks and it orders of magnitude more convenient and betteruse such proper low latency High band local ETHPaul

author: EasyGoing1
author_rank: Regular Contributor
timestamp: February 25, 2021, 02:01:05 pm
content: Quote from: PKTKS on February 25, 2021, 01:41:16 pmBut my question is ... when did fault tolerance EVER become necessary on a home network? I mean that's stuff that Ive used on high availability networks or when being without Internet access for even one minute is not acceptable ... but c'mon man! This is HOME NETWORK ... what would your fail over even fail over to in the first place? Do you have two Internet connections from TWO ISPs? Still not seeing the light here... so ... enlighten me please.Quote from: PKTKS on February 25, 2021, 01:41:16 pmOK, NOW you've won my heart ... Link aggregation is simply too good to not do if you have the resources and skills. I've done it extensively in switch trunking and a few times at home with a Synology NAS ... and I do love what LA can do ... there's no such thing as too much bandwidth, am I right? :-)

author: PKTKS
author_rank: Super Contributor
timestamp: February 25, 2021, 02:11:24 pm
content: Quote from: EasyGoing1 on February 25, 2021, 02:01:05 pm... this actually evolved in parts as required - THIS IS MY BENCH SETUPI can not consider this a "HOME" although still in home..nowadays I spend most of my time on it..My nephew call that "THE GARAGE"..And his motto is:TIME TO PASS THE CHAINSAW on the CLOWNS ..(he loves BRUTAL DOOM...)The CABLE(s) sometimes need to be rerouted ..sometimes taken apart for several mishaps and reasons..As they are different routed the solution copes w/uninterrupted work..Different redundant cabling is very handy...AIR stations are too shitty to anything besides cell phones..may be tube TVs.simple as that - evolved from my life needs...fast pace and 200% dependable on my workstations...Ready made NAS hardly can be handled the same wayas one made by yourself.. and managed by yourselfReasons are rather simple:- very competent integrated firewall- such handy multi layer interface bonding- proper choose of file systems and upgrades.- memory mapped kernel image...- and a *LOT* of other goodies..the list of advantages is just better every year w/new tech..Paul

author: PKTKS
author_rank: Super Contributor
timestamp: February 25, 2021, 02:18:55 pm
content: Quote from: EasyGoing1 on February 25, 2021, 02:01:05 pmYES this last comment w/a separated conclusion...As by ** CHOOSING YOUR NAS FILE SYSTEM... **You actually have an enormous IMPACT on the BW used and the latencya NAS providing NFS directly oran Xserver providing XDM sessions over SSH ..they just require a fastest and minimum latency BW you can achieve..mostly my stuff is NFS and XDM based...it is more than adequateI am ok with thatPaul

author: EasyGoing1
author_rank: Regular Contributor
timestamp: February 25, 2021, 02:49:35 pm
content: Quote from: PKTKS on February 25, 2021, 02:18:55 pmA few months ago I was contemplating buying some inexpensive hardware like a PC on a board kinda thing ... Raspberry Pi 4 style only a little more robust than that and not with an ARM processor. But as I was looking at my options and the number of different open-source NAS operating systems out there, I realized I was embarking into territory that I didn't want to devote the time to in order to learn what my ultimate best options were given my typically picky requirements on such matters. So just ended up buying a single 8TB external, throwing it on the Macbook pro and calling it good for now ... hmm ... call that a band-aid solution I suppose ...But you seem to be pretty enthusiastic about this so in your opinion, which NAS OS do you prefer and why, and in terms of cheap hardware, what do you think is the best option for running that OS? And think in terms of pretty much just me and maybe a firestick streaming media from it ... using it as a torrent downloader and my expectations generally being pretty high when it comes to moving large files to my storage device. The raw data throughput on my new MBP clocks in here...Vs my external 8TB which is on a USB-3 connection ... these speeds are what we would expect on modern USB:If I could get even one fifth of that kinda thruput on gigabit I would be happy.

author: PKTKS
author_rank: Super Contributor
timestamp: February 25, 2021, 03:29:34 pm
content: TBH . this is hard to answer...Setting up your NAS requires you *ALOT* of skills.My PCs are all bench made ... with recent SATA3 and propersetup bonded interfaces STATIC file transfers typically rateabove 300/350 MB/s on proper RAID aggregated CAT6 linksmay be a bit more a bit less ..Dynamic stuff with great latency like NFS that can easilybe HALF on loaded CPUs .. for cheap common PCs.Anyway 150MB/s over NFS for recent gaming is fine.. on cheap PCs300MB/s for file transfers is as of today as GB sizes are still OKI don't even bother buying specialized equipo for that (Mikrotik and so)standard cheapos are still fine.Anything MS WIN is probably a non sense even tryingas their SMB file sharing shit comes from Lantastic (from the 80s)..And their TCP/IP stack is a tragedy in evolution...I recommend just fine SYNALOGY as a ready made solution..just for out of the box working..FreeNAS based resellers are also a very good choice..they brand a variety of hardware... and budgets..Don't be cheap on the switches.. as they can crap your latency timing.Today we are *FAR* better served from choices than 5 10y ago..Paul

author: PKTKS
author_rank: Super Contributor
timestamp: February 25, 2021, 04:29:55 pm
content: ... BTW... important to mention...THEY ARE COMMON HARD DISKS on RAIDnot SSDs not even NMVes.that would increase our BW (bottleneck are the disks..)BUT TRIPLE (at least) the budget..Paul

author: EasyGoing1
author_rank: Regular Contributor
timestamp: February 26, 2021, 01:42:27 pm
content: Quote from: PKTKS on February 25, 2021, 03:29:34 pmMAYBE! Depending on the number of NICs in your aggregated link, and assuming the device sending over the wire has nothing else going on at the moment during file transfer.I don't have the room for a home-built clone for a NAS right now. Really just want something with a small footprint that I can shove three drives in for RAID-5. I had told myself that I would most likely end up with a Synology because I was having a hard time finding something that met my requirements otherwise.Quote from: PKTKS on February 25, 2021, 03:29:34 pmI've rolled out multi-million dollar Cisco networks with voice VLANs, fiber links spread out all over the city, fail-over Internet connections etc., I've been a Network Engineer for over 25 years now... I certainly know "best practices" where switch hardware specs are concerned, and Linksys isn't on the menu, know that!;-)

author: PKTKS
author_rank: Super Contributor
timestamp: February 26, 2021, 05:29:39 pm
content: very good ..for a bench CISCO and Mikrotik are overkill...unless you have some spare ones lying around...The cabling is the most annoying part ...but certainly having a 5~10 wire mesh spreading yourservers will save you a lot or problems when justdisconnecting a simple switch ...you are still working fine with the FAIL LAYER.. althoughwith less than HALF capacity.. your are still onlineToday dedicated switches can do bonding and aggregationmuch easier than before... much much easierTIME will be your most consumed resource setting up your own NAS....After that ... is very easy to keep things just rolling..Mostly because today we have things like BSD *NIX and somededicated distros like FreeNAS already ready to deploy...If you ask me if is it worthy? YES definitively worthyI can keep my NAS much easier than any alternative.rolling as needed

author: 1xrtt
author_rank: Regular Contributor
timestamp: February 26, 2021, 07:48:45 pm
content: Over the last year I grew fond of these Proliant Microservers:https://www.ebay.com/itm/HP-ProLiant-MicroServer-N40L-AMD-Turion-II-N40L-1-5GHz-4GB-4-Bay-3-5-No-HDD/265067343435?hash=item3db73eae4b:g:dOwAAOSwxeNgN9pPAlthough they have an old, low performance CPU, they can be found reasonable cheap, are not too big and, most important, for a home envirnment: Are not power hungry and are SILENT.You´ll not be able to mine bitcoins with it, or use as a virtualization platform, but for the purpose of a NAS Server it is quite good. I use it to backup, media server, torernt and also run a wiki inside a docker container.Of course, as PKTKS mantioned, the biggest investiment is in time, to set up and maintain.

author: EasyGoing1
author_rank: Regular Contributor
timestamp: February 27, 2021, 01:20:19 pm
content: Quote from: PKTKS on February 26, 2021, 05:29:39 pmTrue, but at least Microtik is AFFORDABLE compared to Cisco, and they make a decent product. Sonicwall use to be viable in a more generic firewall space but they sort of evolved into the 'cloud on your premise' kind of device. They now rely heavily on Internet-connected technology for a lot of their features ... which most home users would never need in the first place.Personally, I use a Netgear R7800 and I compile my own flavor of Open-WRT. It's so nice to be able to compile a firewall operating system so that the kernel only has to deal with the features that you need. Makes for a nice efficient router that only does exactly what you need it to do ... nothing more, nothing less. And the R7800 has excellent performance.Quote from: PKTKS on February 26, 2021, 05:29:39 pmLike I said ... 25 years in networking has taught me to use WIRED networking at all costs and only use wireless when absolutely necessary. Mesh networks can actually be a bottleneck if they aren't implemented correctly. Radio waves are tangible energy waves and when you have a bunch of them within the same space close to the source (meaning if the radios aren't spread out far enough apart), they can and do affect each other. Even two radios broadcasting on the same frequency with the same SSID with the same password etc. ... when they are within close proximity of one another the effects I'm talking about can significantly slow down the performance of that network, and radio waves to CRAZY things sometimes ... they can reflect off of surfaces and even become amplified in the right situations where you can have a weak signal, but then move 5 feet away and get an extremely strong signal. I ran into something like that a few years ago at someone's home ironically ... with the WiFi radio sitting on the kitchen counter, when you went 40 feet away into a bedroom, you couldn't even stream video from Youtube and the signal strength vacillated between 15 and 20 percent. But when I went out to the back yard and walked to the end of the property which was about 1,000 feet away from the house ... down past the horse stables and then into a shed that stored horse feed, I got a signal at 80% ... and the kitchen area of the house wasn't even visible from there.That's why I have a blanket rule when it comes to implementing Wifi where consistency and reliability matter ... get a ceiling mount access point and mount it on the ceiling as close to the center of the area you need it to work in as possible. If you need to reach areas that turn out to be out of range for that access point, then get another one that is also a ceiling mount radio and use cat-5 or cat-6 cable to connect those radios to your router and turn off the router's radio... let the ceiling mount radios handle the traffic. Because the bottom line is ... the higher up in the air that you can get a WiFi signal to be ... the better your experience is going to be with that network. These devices that are boxes meant to plug straight into an electrical outlet usually end up being so close to the ground, that they simply won't reach very far... the biggest problem with implementing home wifi is not having the signal source at the ceiling level. If you have an all-in-one router, then get some screws and wall anchors and mount it on a wall as close to the ceiling as possible. You'd be amazed at how many times that solves people's WiFi problems.One time an aerospace engineer took issue with my rule saying that radio waves are radio waves and whether they are close to the ground or up in the air it should not matter. I said, I agree with you in principle, however, I've been doing this a long time and I know what works because I've done it. And also, commercial radio stations don't invest hundreds of thousands of dollars to get their antennas hundreds of feet into the sky for no reason ... that's how the signal reaches people ... radio waves are literally magnetic energy traversing space ... they are susceptible to all kinds of damage, interference, warping ... you name it ... there's enough crap on the ground to say that the best way to overcome it is through elevating the source.This new trend for mesh networking is just a shotgun solution that can and does work, but it's only adding more radio waves into the environment when the best solution would be to have less radio waves but have them elevated higher than people think they need to be.Quote from: PKTKS on February 26, 2021, 05:29:39 pmI think you said that you're a fan of EXT4 file systems? Is that just because they are tried and true and available in any *NIX flavor OS? Or are there other reasons for that preference?

author: PKTKS
author_rank: Super Contributor
timestamp: February 27, 2021, 01:38:08 pm
content: Quote from: EasyGoing1 on February 27, 2021, 01:20:19 pmYes correct.All my workstations are exclusively on ext4 since ... 3y ago more or less.Reasons are a very long list of things that are "upgraded" on rolling base.I started such BENCH setup (and the NAS machine) over an ext2 file systemway back in late 90s... after ditching a SCO server.. replaced by a Slackwarelinux circa 93/95...Very modest thing just to have a sane reliable storage resource for my stuff.Things evolved.. *very* and very fast .. for a long time the rock solid optionwas (and still is although phased out... ) ext3..Reason being that the NAS is a busybox memory mapped image whichallow me to operate the stations even disk less. (in memory). Such memoryconstraint leads to the obvious path: small, fast, reliable, long term stable..TODAY I have already converted **ALL** my file systems to ext4 which hasproven to be already in proper state for reliable operation even in suchminimal memory requirements..EXT4 is working perfectly and all required tools can be compressedto the service image just fine.Other file systems with some "journal" options or "log" options and"compress" options and "rollback" options .. they just are not that readyfor such assumption.So the KERNEL is linked w/all EXT4 and NFS required stuff..leaving the rest as modules just on demand..And i have never ever required other file system...Surrounding stuff like ISO and UFS are not really required asthe memory image boots from non removable media..NFS and NBD are the 2 most challenging ones because theyhave a HIGH LATENCY.. but with a switch with good port trunkingI have never noticed that...XDM sessions over SSH are working transparently ...and only very very very slow end machines would have latency issues.Hope it helps to see how things just were "crammed" last yearsin such setup...I am gradually replacing the master root media w/SSDsand letting only the grunt part to HDsEXT4 works absolute fine under SSDs as long as you use swap on HDDs...Paul