Sunday, November 26, 2023

2023-10-07 - RAID 5 SMB File Server running Ubuntu Server OS

Just so you know from the outset, anything underlined is more technically focused for the project and what I went through to make it work. Anything not underlined is more of the drama involved in allowing me to get the technical stuff to work. I had wanted to do a File server of my own for fifteen years. I have wanted my own Dropbox replacement for ten years. And I have wanted to do a RAID for five years. 

I went into this preferring RAID 10, with striping on at least two drives, which is RAID 0, and mirroring on at least two other drives, which is RAID 1, but they're both put together, making them RAID 10. Suffice it to say that I have a basic understanding at this point, speaking as my future self. Basically what I am saying here is true, but it is more complicated than this. There are other RAID configurations, but I've had little interest in them. I was aware of RAID 5 because of how strange it is, but then when I told Doug, my instructor, about this, he told me that in order to do this affordably, I should do a RAID 5. I hated the idea. 

I only had a few weeks to get the parts for this, some ideas and decisions had to be made. I had one four terabyte HDD which is nearly full, and it has most of my stuff on it, and so it's time for an upgrade. But I don't want to buy another drive that I will then put everything on just to have a bunch of blank drives or multiple drives--I want everything in one place. I have been doing one or the other for fifteen years and it's either a huge waste or it's frustrating to keep jumping from one drive to the other to find things. Of course when I moved away from older drives, they have also aged considerably and are not very trustworthy anymore. So what can you do with that? I need a solution that lets me use all my drives and have a lot more space but keeps everything in one volume so-to-speak. 

I thought I could obtain a computer to turn into a Windows server through my friend and roommate, Nate, who builds old computers and sells them for cheap, ready to go. 

I also thought I had one week to figure out the parts for my file server, and I had an idea. I needed four drives, preferably with more space than I already had available to me now, so two drives for that and another two for backup with RAID 10. I didn't want to put this all on a credit card, so I turned to Nate, and then I had an idea. I have a now old and nearly full 4TB drive, and then I have another 4TB drive to back it up through Windows Backup but it's only a year old. Then I have a bunch of much older drives of various sizes that I don't trust. 

If I buy the next size up from 4TB to 6TB, then I would have to buy four of the next size up and it'd take me a while to get the money saved for that and what I have that I do trust is wasted. So I found another 4TB drive online for about $60, and then had the idea, what if I did a RAID 5 like Doug said, and bought two 4TB drives to go with my one 4TB drive and then also the 4TH backup drive. 

I started putting my plan into affect, had Nate start looking through his usual channels for parts to build a file server, and the only features this computer needed to have was that the motherboard absolutely had to support a hardware RAID. 

When talking to Doug about this project, he absolutely insisted numerous times and in every way that I really didn't need performance out of this computer. 4GB of RAM was plenty...in fact 2GB would be plenty. After all, we're going to do Linux, and Linux is light, free, and works great. He sort of made that decision for me. I wasn't against it, but I am pretty much a Windows guy. 

Apple is proprietary but works however is not very flexible, and is very expensive. Not really meant for gaming, and is mostly geared towards productivity and home use. Windows is not proprietary, very flexible, just works much of the time but not quite as much as Apple, but is more financially accessible, is really geared towards productivity, flexibility, gaming, and can pretty much do most things if not all of them. But you have to pay for Windows. Linux is open source, which means that the masses have access to the source code and can download it and manipulate it if they desire, you cannot do this with MAC or Windows, and it is free. It is also way lighter than Windows or Mac, and is highly flexible and customizable, but it doesn't always just work. In fact if you get into Linux, you will be doing more troubleshooting on average. But as far as I can tell, there isn't exactly anything you can't do with it, except maybe gaming to the extent that you can with Windows. You can still do it, it just requires more work a lot of the time. 

A sort of saying is that with Windows, things just work but you don't know why because you don't have to; Windows does almost everything for you. There are tons of background things happening that you don't have to understand to use it. And then with Linux, you will know everything about how it works because it doesn't do everything for you, and so if you are a real Linux person, you know everything about how and why it works and so you know many things that a Windows guy won't know. 

If you tell me this to tell me that I don't really understand how a computer works, I don't care. My energies are simply focused somewhere else. However, I am not against using it. Give me a compelling reason and I'll foray into it. So foraying into it, I am. I don't have to pay for Windows Server OS because Linux is perfectly well suited for server tasks just like Windows. And it's not like people aren't constantly doing things with Linux such as File Servers, Linux is built with this in mind so no bending is necessary to use Linux for this. 

I went about getting the parts together. Doug, my instructor, said in no small detail that I could even get away with using an old computer. I could use a Core Duo for this. All that really matters is hard drive space and the RAID controller, or the board I purchase that has one built into it. This can be a bit tough to get a hold of. It is common for computers to have RAID built in. Doug made it sound like about half of the motherboards out there have RAID controllers but I quickly learned that just because you have a RAID controller or on the motherboard, doesn't mean that you have the ability to use any RAID option you want. It might only do RAID 0, 1, maybe it'll do RAID 10, maybe JBOD, which means "Just A Bunch Of Disks", and then less common is RAID 5. As of yet, I have not seen RAID controllers in my personal search at all that contain anything other than those five options. There's a RAID 6 for example, but I sure didn't see anything that had it. 

And also, in my search, I was hard pressed to find affordable RAID cards regardless of what RAIDs they supported. 

I should explain RAID 5, but I'll start with what RAID is. RAID 0 is what they call striping. RAID 1 is what's called mirroring. Those are the two most basic types of RAID, from here it gets interesting and uses a lot of tricks. RAID 10 is pretty much just like RAID's 0 and 1 put together and introduces nesting. 0 and 1 both require at least two hard drives to work, both RAID 0 and 1 write to all drives simultaneously, however they differ in what they write and how. So to understand why this matters, you have to understand that data, when written to an empty drive, will write contiguously, or rather in an unbroken, sequential fashion. 

Imagine a round food plater with lips on the edges, and you have peas filling the platter in unbroken circular lines in an outward fashion starting from the center. And the more data written, more peas are lined up in concentric circles outward toward the edges. That's how ones and zeros are arranged on a typical mechanical, magnetic hard drive when it starts out completely empty and fills up for the first time. And lets say one program requires seven peas, they would all be written sequentially in a row with each other.

When you have a RAID 0, picture you have two or more food platters, and they all start out empty and and an equal amount of peas start lining up from the center outwards on each platter. However, the data is broken up. Lets say a that same file that requires seven peas is being added to all the platters at once rather than just one drive. They'll be split up equally among all of the platters, still in concentric lines forming more and more outwards to the edges of the platters. If they were on one drive, the peas that belong together for the program would still be in an unbroken sequence with each other, but you're writing to multiple drives in RAID 0 simultaneously. And because there are more platters, there's more space. Why would you do this? Performance and or extra space. If your drives each write at a max speed of 6Gbps (gigabits per second), and you need them to go faster, you can implement a RAID 0, and all data that gets written is split up equally so every single drives maximum speed is combined with all of the others. 

However, someone thought that this could also be used for another benefit. If you trade the simultaneous operation of two drives for example, and instead you have two platters of peas being filled, lets say you have the seven pea program written in that same old contiguous fashion on a brand new, empty drive on one drive, but every write operation or pea that gets added to one platter is duplicated on the other platter. Instead of related peas being split up between the platters, the peas themselves are being copied for exact duplicates and lined back up in the same unbroken chains of programs on a second platter or drive, called mirroring. This is great for backing up a drive with exact duplication in case of one of the drives failing. 

Also, the spinning disks in magnetic drives are called platters, and instead of peas, ones and zeros are written and read using control arms that have fine points that either use magnetism to write data or sense the magnetic signature that was written so it can be read. And then multiple platters are stacked on a spindle and there are multiple control arms that move across the surfaces of each platter, and on both sides of each platter. 

So that's RAID 0 and 1. Then you get things like RAID 10, which has sort of been my favorite because you get the space benefit of multiple drives being written to simultaneously, striping, and each of those drives also being mirrored at the same time. You have to have an equal number of drives for RAID 10. 

Someone was looking at RAIDs 0 and 1 and thought, why can't we put them together and get the benefits of both in order to have redundancy and space and performance all at the same time? However, this is also where things start to get confusing. The second you introduce multiple digits into a RAID such as RAID 10, you get what's called Nesting

Nesting is where you have a RAID, like RAID 1 for mirroring, which nests a RAID 0 for each of its striping drives or vise versa. So essentially, RAID 10 is very much is a RAID containing a RAID. And then it gets way more confusing because you can have a RAID 50, 60, 100, 150, two digits introduces nesting, and three digits nests a RAID within a RAID within a RAID. This is easy to understand on the surface but to understand the concept of nesting is a bit to wrap your head around. I was aware of these different RAIDs for years and had no idea what nesting was, I didn't even know there was a RAID 50. But there is. And if there's a RAID 7, and a RAID 5, you can have a RAID 75 or even a 750. 

It seems really convoluted and you will wonder why in the world anyone would ever do such a thing, but somewhere out there, there's someone doing things like this for a good reason. Perhaps you want triple redundancy but like me, you don't trust all those layers because they have a potential single point of failure despite the layers of redundancy. But redundancy comes in all shapes and sizes and for different reasons and can be shaped in the case of nesting in particular to suit a specific situation. Consider that 0 is fast, but 1 is no faster than a single non RAID drive by itself. But you're getting a benefit somewhere. If I want the benefit of striping but also of mirroring, then nesting is useful. If I want to use RAID 5 to save money on the number of drives needed, or I'm willing to take a chance on RAID 6 with the ability to lose two drives instead of just one drive with 5, but I then start making more money and want more redundancy but I merely want another RAID 5 or 6 to mirror my existing RAID 5 or 6, then I can do a RAID 51 or 61. You can go as high as triple digit, and the only used example Doug has ever seen is a RAID 100. People can use that if they want lots of speed because you're doing two layers of striping.  

Man I hope I'm not blowing smoke. I already explained a ton of this once and realized I was wrong and threw my whole explanation away, to then do this far more elaborate explanation minus the picture I drew to explain what I was saying the first time. 

But there are several pieces still missing to this explanation. RAID 5 finally, is where you have a minimum of three drives. 

It just occurred to me now, and therefore I had to look it up, that all drives in RAIDs have to be equal size, although you can use different size drives in a RAID, they will simply all be set to the size of the smallest drive. So if you have three drives striping with each other, one of 500MB (Megabytes), another that's 1TB (Terabyte), and the last one is 2TB, then they will all only be used for up to 500MB, and to do otherwise would create all sorts of problems. So if you want that space, you really need to have all drives of the same size. 

Moving on, in a RAID 5, you get what's called parity. This is even harder to explain for me, but to me parity and nesting can be equally as confusing. If you don't want to go into the specifics of how they work, then they aren't so terrible to understand, but that's if you ignore how parity or nesting actually work. So RAID 5 requires at least three drives but can take any number of drives more than three unless you run out of ports to plug them into or something.

Lets say you now have at least three food platters, and you add a pea to the first two drives, those two are data that would be written sequentially to one platter, but they're again split over two drives, although I wouldn't necessarily give RAID 5 the performance badge, when you fully understand RAID 5, you'll see that it can actually be quite slow, but not necessarily in the initial writing process, just in the parity bit. So you have three platters, peas written to the first two platters,, and then the part that takes such a long time, the third pea that is added to the third platter is what's sort of like a fake bit, it's not data that can be used for anything other than the RAID because it isn't like a picture file you can use, and it's also not the backup data which would also be like a copy of the picture file you can use. Instead it's a parity bit or parity pea that uses complicated math to determine what one of the other two peas would be if the first or second platter was lost. 

In this case, lets say you lost the second platter and you replaced it with an empty platter, the RAID will use the information it has based on the one legitimate pea that's in the first platter and the parity pea in the third platter to figure out what kind of pea was in the second platter, and then recreate it from that available data. This works because we're using binary, ones and zeros. Lets say you have a one on the first platter and a 0 on the second platter and a parity bit on the third platter to represent those other two bits on the first two platters. The parity bit that gets written to represent those first two bits is determined by what bits are in those two platters. The last platter only gets one bit to describe what was there. And this may not be how it works, but since it is either a 1 or a 0, what it probably does is if it is a 1 on platter one and a 0 on platter two, the parity bit will probably be a 1 to represent an odd number because one of them was a 0 and the other was a 1, one of the platters was lost and so based on what it has now, if there's no 0 and there's only a 1 on one of the platters and then a parity bit on the third platter that says it was an odd number, then when you add a new drive to replace the lost second drive, the available bit in platter one and the parity bit will have to equal an odd number, so in my example, it'd be 1. and so the RAID controller will know that this missing bit was a 0 and write that bit to the brand new platter in order to rebuild what was lost. 

To further explain other scenarios, if its the first platter that was lost, then it would know to replace it with a 1 because the parity bit demands the rebuilt data be an odd number bit and so if platter two has a 0, then it knows to replace it with a 1. If there are seven platters, this still works because say each bit or digit in this case I list here to represent what bit is on what drive in this example is on a different platter: 101000. If you add up the two ones, that means the parity bit is a 0. So if any one of these platters or drives is lost, the RAID controller adds up what bits it has in the same positions on each drive, adds them up, and assigns a parity bit in this case of 0 because it was an even number, so if one fails and you replace it, how can you know what was missing if seven platters each with one bit have to share only one bit to back each of them up, it knows that if one of the drives for example that was a zero is dead and it still has a 0, then the missing bit must have been a 0, if it was a 1 that was lost, then the parity bit will still demand that there should be an even number so it would be zero and with one drive down and it has the correct even or positive number, then it knows that the lost bit is a 0. So no matter how many drives you lose, this still adds up because RAID 5 will always only ever be able to lose one drive because it requires all drives to vouch for what any one failed drive has in conjunction with the parity bit. 

This method of error checking or bit checking is also called Checksum and is also used in ECC error checking RAM, which again, uses one bit at the end of a series of bits to vouch for what bit is missing by using even or odd number representation. 

RAID 6 does something very similar but the scheme is different, you can afford to lose up to two drives, however the math for RAID 6 is far more complicated than for RAID 5. I'm not done explaining RAID 5. Now we move on to the other part that makes RAID 5 confusing. 

Imagine again that you have three platters with concentric circles of peas written to them. 

So you see that the backup data used for parity is not real data, it's not a copy of the lost data, it is literally just a bit that says that because this bit here is this and I have a representation of the sum total of bits in the same position on each drive adding up to this one bit in binary, it then uses that to recreate the lost data. So even though the recovered data may be a perfect match for what was lost, it was technically created through a very complicated process from scratch and your original data is actually gone, but it was able to be brought back from the brink by surrounding data that could describe the details of what that lost data looked like. 

It's very much like a thief in the night leaving evidence of their presence, the footprints tell us what size shoe they were wearing, possibly what kind of shoe by the tread pattern, where they walked in the house, were they running or walking based on the distance between marks and characteristics indicative of a print made while running, and by examining all this information, a forensics expert can recreate or imitate what the actual thief did in order to get a clear picture of them in the act. They can demonstrate what shoes they had on, how big their feet were, whether they were walking or running, possibly how tall they were by the length of the stride even, the characteristics of their gate as they walked, how many times did they go to that one spot and how many times might they have been standing there because numerous prints were smudged over each other as they shifted their weight and certain prints got deeper and darker than others and they re-stepped in one spot so the new print is slightly off from the original print before it. And so the expert can show you exactly what they did. They are not the thief but you know what you need to know to recreate the data of what the thief did, or in the case of our parity, what bit was lost, so that it can be imitated and written to the new platter or drive even though the computer can't know wat the file was with all the bits put together, it can rebuild the file with the cleverly used trick of pseudo data, bit by bit. 

Just to be clear though, as I realize the flaws in my explanation, RAIDs do not apply to individual platters in a single drive encasing, they apply to multiple drives only and each encasing with its own spindle and platters are treated as one drive. The platter explanation is merely intended to explain how data is written to a disk by using the whole disk, encasing and platters combined, to write on a hard drive in various conditions. 

Moving on!

I had many conversations with Doug about the reliability of software RAID (SRAID). I had learned six years ago that you should stay away from SRAID and stick with hardware RAID (HRAID). There were many stories about how if your OS goes down or something happens to the motherboard or this or that, then you lose everything, plus HRAID is faster. But I had many debates with Doug where he assured me of how great SRAID is and that it can be backed up and recovered and this and that. I just need to save the configuration file, which is apparently only 1 KB. Hard to believe that this is all that stands in the way of great loss, a 1 KB sized config file. But despite multiple debates with Doug, he would explain circles around my argument, but still leaving me feeling unheard. However, I felt the only way to rectify this would be to ask my friend Bo, who has been in IT for twenty plus years. Seems like he always has some point that doesn't jibe with what Doug says, so this has to be ironed out before I can even consider trusting an SRAID. Seems that no one in my class or my instructor Doug has ever heard of this reliability concern with SRAID so maybe Bo knows...ha ha, Bo-Knows. 

He told me once he was thinking of doing a podcast and calling it Bo-Knows, but I guess there was a name copyright thing or whatever so it never happened. 

While this RAID debate was in full swing, Nate discovered the old, used motherboard he had located and I had just purchased had a few bad capacitors. I panicked and Nate offered to buy the old motherboard from me to make it up to me since he missed it in the purchase. I didn't do that because I had more money than him. I was miserably depressed because my project was hanging in the balance and I couldn't wait to get it going. I didn't want to spend more money to get a part I already had. I was already looking at trying to buy a power supply and couldn't find one. 

I ran the power supply issue by Doug and he recommended that since I work at Deseret Industries, a thrift store, they get computers in all the time, and I could buy an old one for five or ten bucks just so I can steal the power supply that would ordinarily be far more expensive all by itself. I looked, I found none at work. I checked another location, no dice. 

Nate had a power supply he bought for $50 or so, and never ended up using it, and it had far more wattage than I needed and I preferred modular or semi-modular power supplies. I prefer having not having the sum total of all power cords that can be connected at the same time, most not being used, get in the way, are eye-sores, complicate cable management, and exit the power supply in a huge bundle. If I just need a bunch of SATA power and no Molex, I'd like the option. 

I found out the hard way that Modular is way more expensive, and then to add insult in injury, you really only start to see modular power supplies around 600 watts and up and I was thinking more like 500 or less, thinking I could save money by searching for a less powerful, lower quality PSU, but I will also have to sacrifice my love for modularity in order to buy a PSU from a trusted brand for no more than $40, and $40 was already as high as I was willing to go, finding only two brands I trust. 

I had this idea after a mostly depressing day at work, feeling down about my capacitor predicament. I suddenly thought, wait, this could just be apart of my project, replacing capacitors in class. I could add this to my list of experience with this project. Not only do I build a server, make it functional and use it regularly, learn something else about Linux, have another computer build under my belt, set up an HRAID 5 on it, and then write about all of it in this blog, but I can now add soldering as well as replacing blown capacitors to my repertoire. 

I ran the idea by Doug. He shot it down nearly instantly by telling me it is likely to be too specialized for this class. I asked, wait, we've got a soldering iron in class, what do you mean it's too specialized? He said it was likely too small. 

So now it's Groundhog Day, reliving my misery at work again. Out of desperation, I asked Nate over the phone how hard it would be to buy another board, when he said he'd start looking, and suddenly I thought to ask, wait, don't you solder occasionally? He said yeah. I was beside myself. Why hasn't he offered to solder this thing? I have a soldering gun if that's the problem. He said he could do it, but he needed the right capacitor and more flux, which I would have to buy. "Okay, how much damage is this?" I asked. He spoke as if lowering the boom, and I thought he was about to say that I might as well just buy a new board when suddenly I registered in my head that he said about $10. Man, that's it, lets do it! 

I thought of the reverse of that scene in Father of the Bride when he gets the news of how much his daughters wedding is going to cost with a wedding coordinator and he misheard the guys quote, he thought for a second there that he said one hundred and fifty dollars per person that attends the wedding, and the guys like, "No, noooo!" "Good I was about to kill myself!" "No, two hundred and fifty a head." 

So Nate was willing to solder a new capacitor. There's a major downside, I get it fixed and don't have to buy another motherboard, but if Nate does the fix, then I will not be involved in the soldering job and get this extra experience. That thing that excited me several days earlier about this really unfortunate occurrence having some sort of upside because it can now just be apart of my project to fix it, was then removed from the equation. Nate hates showing me how to do things or helping me do things. I have a tendency to be particular, and he has a tendency to just get things done even if they're substandard. I asked him if he could at least show me what he was doing while he soldered so I can at least write about it in the blog and he said sure but I'm not going to see anything because his face is going to be right up in there, working really closely and I won't have a view. I knew he wouldn't explain what he's doing. So eventually I gave up and just let him do it without me. What was I going to gain from being there? At least the capacitors would be repaired. 

I thought about this, why did Doug think this was too specialized? Nate was about to do it next to his keyboard with a cheap soldering station while watching the Simpsons. He then moved it to the kitchen table so I could watch until I wasn't able to show up anyway and gave up trying to be there. I took pictures of the capacitors I had just bought and showed Doug the picture and asked him what was so specialized? He said, oh, I thought it was one of those capacitors that are no larger than a few millimeters long, no larger than bread crumbs. 

It didn't dawn on me to move the soldering operation back to class, Nate was already about to do it. 

Then the night before Nate was going to solder the capacitors, I asked Doug on the way out the door after class, wait, you still sounded a little apprehensive about helping me replace the capacitors. You said you still wouldn't help me do anything but remove them, I'd have to solder the new ones on myself or something to that affect. He said, oh yeah, well, your friend's about to do it. I said, wait, yeah, but you knew that when you said you wouldn't do anything more than remove them. I was confused. Why say anything if he knew it was covered? He said, well I don't want to screw up your board. I might mess up and you'll blame me. That seems to imply that this is risky. Should I not have Nate solder my board? He said, sure, if he knows how. Well, you know how. Yeah but its risky, he said. I could accidentally burn the PCB on top of trying to remove the capacitor. Oh. 

Nate started working on the motherboard while I was out and ran into a few problems. He was able to remove the capacitors, but the contact wires for one of the capacitors wouldn't melt and we couldn't get the soldering iron right on them, and we needed to remove them in order to put new ones in. I got home just in time to see this unfolding. Nate and I worked on this for about fifteen minutes when I thought, why don't we just heat it up as much as we can and poke the thing as hard as we can? Do we have anything as thin as these contact wires and strong enough to poke the old, un-melted wires out? I asked for the bag of new capacitors. I grabbed a new capacitor and heated the hole as much as I could and started poking. 

I was able to poke the contact wire out, but only for one of the two blocked holes in the motherboard. I had already been on a rollercoaster ride over this, I got paid that day and I was due to bring this computer in the next day to start working on the file server in class. Not wanting to waste a good Thursday project day because there was only one project day a week to work on this in class, I wanted this problem behind me. I decided to go buy another motherboard so we didn't get hanged up any longer. 

I got another used, older motherboard. But this time it was considerably newer, maybe seven to ten years newer. This was an MSI 970a-G46. 

Turns out I had the g43 for 9 years and had just rebuilt my computer several months earlier and gave the old one to Nate as payment since I suspected many parts of my computer to potentially be at fault for a video cut-out a few seconds long every few seconds. I discovered the irony and found out that my old board had the same raid controller that this other board has. Could have avoided this whole charade if I had just kept my old board. So the second used motherboard was now installed in a case I was borrowing from Nate, along with that PSU I was also borrowing from Nate just so I can get this project rolling in class. 

Day 1, a student named Jacob was helping me and we began working on the file server project with the new motherboard. We set about to figure out what to do first. I looked up a video on Youtube that kind of helped us to figure out that turning RAID on in BIOS was the first step. So to access BIOS, I think I mashed and spammed F1, F2, F12, Delete, and Escape. Which ever one it was, it seemed to work. We scoured all of the options in the BIOS to find it. While in Standard mode in my BIOS, we went Settings > Integrated Peripherals > and changed SATA Mode to RAID Mode. 

Turning on RAID Mode seemed to work totally without a hitch, so we tried to access the RAID controller from there according to that same video I found on Youtube. This started to become very difficult. Other instructions I found online said specifically to use Ctrl+I or Ctrl+M. 

I didn't know it was called the RAID controller yet, or that it was located in the North Bridge. I thought it was just another thing similar to BIOS or to Recovery Mode, just stored in the same place. I also had no idea that the chip in the middle of the board, below the processor, which often had a heat-sync on it, was the North Bridge. Apparently a lot of features are housed there. And apparently there used to be a South Bridge but I think that's not a thing anymore because it's unnecessary or we moved away from that type of architecture of something. 

We entered the raid controller and selected raid 5 and selected all disks and the raid 5 was created. However we were looking over our settings and were confused that the RAID configuration was only recognizing maybe 5 or 6TB of my 16 available TB of hard drive space. It turned out that it was only assigning each 4TB drive a total of 2 usable Terabytes. Doug took the reins and went back to BIOS and saw that it was in legacy mode and he was like ah ha or something like that, switched it to UEFI and then had us re-enter the RAID utility to continue with the HRAID. But now we couldn't access the RAID utility for some reason. We would enter Ctrl+I and Ctrl+M and nothing would happen. 

We tried to figure out what the problem was but we were stumped. We went back through the BIOS, we rebooted, we turned RAID off and back on again. This BIOS has the option of switching at the top of the screen between ECO mode, STANDARD mode, and OC GENIE II mode. I tried going through these different modes just to familiarize myself with them more fully in case there was anything I was missing. That wasn't helpful. After some time I got back to STANDARD mode. We probably tried different SATA ports. 

Some time later, Doug came by and went through the BIOS to see if there was anything he could find. We went through all menus he deemed at all relevant to creating a RAID. We got back to the Settings menu, searched, and then again clicked on Integrated Peripherals and looked around. He spotted the option right below the one to switch between SATA mode and RAID mode, called Board SATA RAID ROM, which was set to Legacy mode. He had us click on it to discover the option to switch to UEFI. That turned out to be important and revealed a far larger problem. 

It turns out that Legacy mode means using MBR. So MBR, or Master Boot record, is a partitioning table and when they first created it in like 1983, they were thinking that they needed a standard that would allow them to partition a drive, or rather, split a single drive into multiple containers. Drives in those days were really small and they never imagined that files and drive sizes would skyrocket, so they said, lets just make the maximum partition size two million megabytes and a maximum of like four or so partitions. Great how that worked out for us!

So I think the problem here is obvious, we're using MBR in legacy mode BIOS and that will limit all four of my 4TB drives to only 2.2TB. So it needs to be GPT. GPT, GUID Partition Table, which was created in the late 1990s, recognizes far larger partitions as well as allowing for far more of them.

As a quick aside, I thought I knew the difference between BIOS and UEFI. I thought UEFI was newer and replaced BIOS, which apparently unless I'm wrong, didn't replace BIOS, it just adds to it or something, they run at the same time. I thought if you had mouse functionality, then it was UEFI because BIOS doesn't have that sort of thing. I guess I was wrong. 

So the way Doug explained it to me...I only seem to briefly understand things when Doug explains them. I seem to know stuff but if I explain it, I am always wrong by a longshot. I suppose I could just never explain anything and people will always think I'm smart. And I'll seem mysterious. lol. 

The way Doug explained it was that depending on how new the motherboard is, what you see, the graphics, all the stuff that makes the BIOS or UEFI look fancy is just a result of the time that the motherboard was built in. They both work differently, the thing you see which we call BIOS or UEFI is just an interface, but it starts to do its stuff when you start booting. And you want this process to be backwards compatible so I assume all new machines have BIOS and UEFI, and can be switched between the two depending on the situation. However Windows 10 and later are UEFI only. They cannot run on BIOS. 

So the beginning of the far larger problem slowly starts to reveal itself here. Doug seemed confident that switching from Legacy to UEFI solved the problem we were having in getting the RAID controller to work. He already knew why Jacob and I needed to use UEFI so that we could do the RAID with GPT. None of this meant anything to me yet. So we went about the process of re-entering the RAID controller. We couldn't get in, and it would just skip on to installing the OS. We were confused. It was working before when in Legacy mode on BIOS, but now it wasn't working in UEFI. We would get to the screen in the boot process where we had to either press CTL+I or M, and it wouldn't respond when we did so. We worked on this for quite a while, Jacob had to go because it was the end of class. But before I started packing up, Doug was actually very curious about this problem and continued to work on with with, wanting for me to be able to make some sort of progress since we only get to do this once a week. 

Basically we spent the next few hours trying to do a firmware update. We found something online for this motherboard that said that there were RAID set up files for this motherboard. 


So we tried to do a firmware update. We talked about updating the BIOS, but this made it look like it was specific software that was supposed to come with the motherboard that would allow us to do a hardware RAID or HRAID. So the RAID controller only worked in legacy mode. This really annoyed Doug. As he would later put it, "Why would you put out a motherboard with all these features, just for none of them to work when you switch to UEFI mode!?

Doug recommended I do software RAID, which I didn't want to do because my understanding was that SRAID was unreliable. 

I wasn't ready to give up on HRAID. My roommate, Nate, was now also helping out in looking for the RAID setup files online, and he found a few places, so we downloaded those files which Doug and I had tried before we finally went home, but they were the wrong format from what the motherboard was requesting. 

At this point, Owen, the security guard, walked in the room about an hour or so after class to ask how things were going and Doug said oh yeah, we need to get out of here before the alarm automatically activates in the building. Owen said, no actually that activates at such and such time. So Doug and I kept going. 

Doug got an idea to try something, since the internet was telling us the files we downloaded were the files we needed, but the motherboard kept not accepting them and saying that we needed this other weird format Doug had never heard of. He got an idea to just change the extension to the one the motherboard wanted us to use, but he was concerned that this could brick my motherboard. We decided that we could try a BIOS update to see if that would change anything and then we could finally install the files or if perhaps the update would already have the files included.

Nate took it upon himself to update the BIOS for me while I then decided to give the older motherboard another try since we just needed to clear those soldering holes. Doug then instructed me to not go and blow a bunch more money on yet another motherboard or other parts that I was not already planning on buying to make this work.

So back to the old motherboard. Nate and I had stalled on fixing the capacitor on the old motherboard because I just ran and bought a new motherboard, which is the board that Jacob and I were struggling to get working and then Doug and I spent hours after class working on. 

Doug said I could bring the motherboard in Monday the next week and he'd allow me to do a project day kind of thing since he was going to have some time. And I think I was caught up on my work so he would see if he could remove those stuck leads.

I brought in the old motherboard so he could try and fix it himself, he held the soldering iron on there for much longer than I had, and repeatedly, Doug managed to get one of the two blocked holes free but the last one was proving very stubborn. He did the soldering iron on both sides of the motherboard, poked it more, and he turned it up as hot as the iron would go, and used the de-soldering suction gun we had in class. But the blockage would resolidify before we could get the suction on there and neither of us could get a good seal on the board with the suction anyway. We ended up using the sim tray pin tool that came with the iFix-it kit we had in class, and while pushing on it with the sim tray pin from one side of the motherboard and melting from the other with the soldering iron, Doug eventually got the stuck piece out.

The motherboard now had burn marks where the two holes were cleared, but it was fixed. Before I went home for the night, he said to go ahead and bring in both motherboards. Oh and he did also say that there was a good chance that he friend the board and it wouldn't work, however I think he also said that this might not have mattered anyway because for all we know, these capacitors really didn't factor in what we were going to be doing with the board. So this could have all just been academic. 

I bought a new computer case I thought emulated what I liked about cases but also this particular project. It was an open-frame case for less than a hundred dollars. ______________






Project day Thursday, it was before I was going to head to class and I had switched the new motherboard that couldn't recognize the whole RAID out of the computer case and put the old motherboard back in. And then I ran into class with the old motherboard in the new case, ready to hopefully get the RAID configured finally.


The next number of classes was just about doing assignments and discussing with Doug what we could do to rectify the situation. I told him the old motherboard I bought first for $25 was still in play because I was having my roommate, Nate, solder new capacitors to the motherboard. So he did. I was going to watch but had too much going on and so last minute told Nate to go ahead without me. I got home before he finished, just in time for him to tell me he removed the bad capacitors, but part of the soldered parts of the leads from the bad capacitors broke off and were now stuck in the motherboard.




________________________________

  • in class we discovered that the new motherboard had a fake raid controller that had hardware built in for the raid, but it relied on software, I had to decide if I was going to try to make an SRAID and doug kept on trying to make the raid himself to see if he could do it, so he could then delete it and let us do it for the learning experience. He failed. But we had a few options. I couyld bring in the mpold motherboard monday to see if he could clear the soldering hole. 
  • that next monday, i brought in the old motherboard to see if we could clear the soldering hole for both contacts for the capacitor. doug managed to clear it using a plunger tool, some solution emant for difficult solderings, and then he just resorted to using melted solder wire to mix with the broken capacitor wire and we managed after some time to melt it and clear the hole.
  • i ordered the new case. I wasnt going to buy this case because the old motherboard from the HP xw or wx 4400 had a different form factor and I didnt think any new case would take it. But since we bought the new motherboard, I went ahead and bought it. In putting it together nate and i discovered that I couldnt have bought the more perfect case if for whatever reeason I did end up needing to switch back to the old motherboard because all the connection points were totally adjustable sideways and up and down.
  • doug said that it may not be necessary to fix the capacitor because theres a chance that it didnt actually use that capacitor so this could be a waste of our time. 
  • i got the computer case and installed the old motherboard, nate soldered the new capacitor on and we changed the new motherboard out for the old one that we thought had a real raid controller instead of a fake raid like the new one had. 
  • doug did warn me that he was not willing to make the new solder himself because he didnt want to be responsible if it didnt work. I told him nate was helping me do it. Then he pointed out that it may not work. 
  • doug told me i should bring both motherboards just in case. 
  • second thirsday, the 12th, i brought the computer with the new case and the old motherboard in with all the capacitor repairs done to it, nate and i tested it and ot booted just fine. I worked on it to creaet the raid, did so easiuly. but when selecting the disks for the raid on the old motherboard this time, i disicovered that the partitioning scheme this computer used was mbr instead gpt. unfortunately mbr only recognizes disks up to 2TB in size. I asked if we could just partition the disks so they were only 2 TB per partition on each 4TB disk and he said thats the problem, it will only recognize up to 2 TB on a disk anyway so we wouldnt even reach that point.
  • we thought about installing win8 or 10 where gpt is recognized, but we determined that this motherboard was too old in order to recognize anything newer than win vista, which used mbr. 
  • I switched the motherboards in 15 minutes and got it tightened in and ready to use. we went back to trying to solve the problem with the old board. good thing i broguht it. it had gpt. but i did some research on the raid controller the new motherboard had and discovered that it was not a fake raid after all, it was a full raid controller that did not rely partially on software to function. its just that it was integrated into the south bridge and thats why it wasnt immediately distinguishable on the board.
  • i had activated raid on sata mode juts like before and immediatley after that, changed the legacy bios to uefi mode since i saw doug do that last time. but we were totally unable to access the raid cinfiguration utility. we couldnt figure out why. it wouldnt even acknowledge that there were disks connected to the board to install anything on. all the computer would boot to was the bios. we played around with it for a while and turned uefi mode off. all of the sudden we were able to create a raid 5 with all the disks being recognized just like last time. but doug said we had to then go back and change it to uefi mode again because ubuntu server couldnt be isntalled on legacy mode. a little while later we discovered that th e reason this wasnt working was because the way legacy bios and uefi read disks is completely different. So what you  do in one mode, is unrecognizable in another. create the raid five with all disks in legacy and switch to uefi and all of the sudden it couldnt find any drives to boot to or to install an os to. 
  • i was frustrated that no matter what we did, we couldnt get this thing to work. in the middle of this, a game sort of came about organically where every time a group of students working on a project said, "Hey doug" because they needed help, he had to do ten pushups. He was allowed a grace period of 5 minutes so if fve minutes had gone by and someone said it, he had to do the pushups and then come help. I got a littke impatient but never sdaid anything because i needed his help at this point being completley stuck and time was being wasted because he was doing push ups every few minutes before actually coming to help. Then he sat at his desk for a minute while I was waiting and I said "hey D..." and realized what I was doing half way through and then completed my sentence, "...oug" and he gave me an evil look and then did ten pushups. Everyone said I was playing with fire even though  this was my first time making him do pushups. I guess it didnt really help that 20 minutes earlier I made a deal with him that I would just say "Hey Mr. Douglas" so he didnt have to do the pshups. So having made this deal, I made him do pushups and he gave me an evil look. 
  • Class was over, everyone but me and Jack had packed up and left but he was still trying to figure out the problem this other group was having, which was strikingly similar to ours. They too were trying to create a partition and install linux, they just werent making a raid for a file server, but I think they were trying to make a server. They had failed and all but jack left, so doug was trying for just a little longer to figgure out what the problem was. Then jack left when doug still failed and said theyd try again later, and then he came to me and tried again. We were there for another nearly 2 hours before we both went home and Owen the security guard came in and said that just so we know, the building automatically activates certain security features at 11, and it was about 10:30. 
  • we had tried several times to get the raid to be recognized or to do the raid in legacy but he knew that ubuntu wouldnt be recognized, so we looked at firmware. we tried to flash the bios. doug found a page online that looked like it was saying that with this firmware update, it opened up the feature to do raid in uefi mode rather than just legacy bios. we tried to flash the uefi from the bios but it wouldnt recognize the file format the update came in, which was an exe. I called nate and he did this sort fo thing all the time, he found an update that used another format and the vboard still didnt recognize that. doug tried a few more times. I asked what if we changed the format of the update to a .txt file and looked at the ocde and then doug had the idea of just changing the format from the format we had to the format the board regognized which was .ob1 i thin,. yeah, star wars, haha. I didnt realize that till a few days later when explaining it to nate and asking f he could figure out how to flash the bios. 
  • finally doug concluded that we needed to isntall win10 on this motherboard to flash the bios through win10, which would then accept the .exe format the first update he found was programmed in. but we couldnt figure out why it wouldnt isntall win10. then as we were packing up, doug said, wait, lets take the sata ports out of raid mode and see if it installed win10 then. I said my roommate should be able to do that since we werre leaving.
  • nate did it. win10 was installed, the bios was flashed, and it was at this point that I asked nate if that means the uefi was also flashed and he said he didnt know but maybe. i also asked if maybe the uefi raid update could possibly be a standalone update or something and we didnt find anything that indicated any such thing. 
  • talked to My friend Bo Landsem on phone. been trying to get him for over a week and a half to ask him to explain the reliability factors of HRAID vs SRAID. 
    • this turned out to potentially throw a monkey wrench into my whole project because when I explained any aspect of the project to hi, he began to question every aspect, asking why would I use Linux. He has points. Windows does make things easier. As far as the productivity factor because of how much time you spend solving problem's and much of the reasoning to use Linux is because of the money you save because Linux is free / open source. However with Windows you just part with the cash even if it is exorbitant depending on your use case or you just would rather save the money, the fact that it is done so much more simply in Windows saves you so much time and frustration that you have the ability to take care of other issues for that client or other clients, or you can simply work less for just a little less profit. There is merit in the argument. I am personally a Windows guy, but that is only because I grew up using it and understand how it works in a lot of ways. I know what it can do which lines up a lot of the time with what I want it to do. Each OS has its own benefits and disadvantages. But I am trying to be objective here. Linux isnt my first choice but in looking at my project, I do want to keep copsts down and I am willing to learn how to use Linux in order to complete my project and get what I want. Bo made the point that things just work better, they break down less, theyre easier to maintaina nd set up, most of the time they just work, but you have to pay for it however the cost is worth it. His friend Scott and other friends of Bo's make the point about cost, they talk about other Linux features vs Windows features I dont know about I am sure because they have these debates over the course of decades, and Bo mentioned that many of these friends have switched to Windows finally because it is easier and worth it in the long run. Okay. But the point here is to learn and keep costs down. So Bo talked about that. He actually did convince me to switch to windows, but since I want to be a network engineer like Bo, I wanted to do Windows Server, he recommended the latest one and those were easily multiple hundreds of dollars. I thought, well surely I could buy a used windows key for like $30 like you can with windows 10 and 11. He even recommended it which I thought wasd weird because he listed a bunch of reasons why used keys were actually not a great diea back in 2019, such as that you dont know whaen they will expire and so you buy one just to find out a month later that it expired and now you have to buy another one and you just dont know the condition under which they were used, what package were they apart of. Businesses end usage or go out of business all the time and so to make back some of the money from the failure, they will sell everything they had such as computers and equipment, if they owned the building, and in this case, windows keys. But now he was telling me different and so I thought id do that but then he turned me to a site where you could buy keys and the keys were for hundreds of dollars, and may of the keys were just for server cals, which is the purchase of access for more computers to your server. You can buy a server OS with no CALS, or just one CAL, which would be ideal for me because I would be the only administrator and this doesnt necessarily affect how many computers I would share connections to the server with. In my personal case, they could all just use the one password or whatever. I'm not sharing this with more than maybe one or maybe at the absolute most, two other people. But just the CALS were at least 30 dollars just for one. So I thought, okay. back to Linux. I want to keep costs down. I was willing to go the windows server route since I want to work with windows server anyway, it started to make sense, like wait, I am missing a prime oppertunity here to get into the neiche I want to be in anyway, why wouldnt i do windows server OS? And then I saw how expensive they were. Then Bo started attacking why use a server OS at all, just use win10. He explained he could create a server on his computer right now in just a minute or so. I thought, okay, thats interresting, I would like to know that just for achedemic reaosns, but I still want a server. Then he attacked that for a while, no really, you dont need a dedicated compyter for it at all. And I thought I had finally developed the ultimate end argument comment because this is not the first time I have been badgered over the way I want to do things juts because someone else, who is smart and in general, smarter than others around them, think that ebcause they have so many things figured out, it makes no sense why people would conduct their lives or a particular thing any other way than these smart people have learned to and actively do condict themselves. 
      • I had a friend that I often got help with problems with my house or truck, and I explained that I was trying to make more space in the cabin of my truck for stuff so I could clear out the bed of the truck. It was all stuff I want to keep in the truck in case I needed it. But this other friend, Adam, was like, i see you have 12" subs under your back seat. If you just get rid of the subs then you have tons fo space for all that stuff thats in the bed. And I said no, I use the subs and I like using the subs. In fact, I use the subs every time i get int eh car. And I was beside myself because this reasoning meant nothing to Adam, he kept harping on why I dont just get rid of my subs and its like, dude, I want to use my subs more than I want to use the tire pump in my bed. Id be more likely to lose the tire pump than the subs. The punp is there for emergencies or occasional convenience. Okay, so get rid of the subs. Not happening. This just went on for fifteen minutes, and then he escalated his argument, in fact how often do you use your back seat? Maybe once a year I said. Okay, so why use your back seat at all, just hinge it upwards into the storage configuration against the back of the cab and put all the stuff thats in the bed here in the back of your cabin. No, I want to use the seat when I need to. He then made it personal and compared my reasoning to my moms reassoning because he saw her apartment where shew clearly had way too much stuff for such a small apartment, he asked if she conducts banquits because she has enough cookware for it and I said no, she hardly cooks at all. I explained that she lost a bunch of her cookware in a devorce and so shes like someone from the great depression where they can just never have ennough of that thing saved up for the possibility that they go through another situation like that again. If it was food, then you stockpile food, if it was money then you could never have too muc j money saved up and itll be one of your highest prioroties. For my mom, one of those things is cookware. Im not justifying it, but that is the fact of the matter and I have talked with her about it, others have, then they ask me to talk with her and I tell them Ive been talking to her about it for many years and its just not something I have been able to get her to get through to her about. And now adam was using this as an example why I shouldnt keep my back seat and should get rid of my subs. Ok, lets put this in perspective, you have your way, I have my way, I am dissatisfied with my waybbut that doesnt mean your way is THE SOLUTION TO EVERYTHING. There are many ways, and often it takes me a decade to solve a seemingly stupid problem. Like how do you get suspenders to stop slipping off your sweat pants, well, it took me ten years to finaly just put a safety pin in my beltless pants so the teeth of the suspenders have something to bite into. Who gave me this solution? It wasnt Knows-Everything Adam. And guess what, it works. It was simple, cheap and very effective. That is what I look for a lot of the time. In an unrelated note, what did Doug say in class like a month ago when troubleshooting something for an employer, he said always be simple, fast and cheap. ????????????????????????. Then Adam escilated his argument again saying that you should never keep anything in your car because people break into cars, and if they breka in and take nothing, you still have to replace your window or door or whatever. But if you leave it open, and theres nothing to take, nothing gets damaged and nothign gets stolen and if they dont see anything they might want to steal then theyre less likely to try and enter your car. My question is what does he do to keep his family safe in their house? he argued that you can buy lock picking kits at the local store. ge gave an example of this time when his neighbor had locked himself out of his house and adam went to the store to buy a lock picking it while the neighbor decided to climb into his attick and get down to his fron door and unlock it from the isnide. And adam got back from the store and had picked his front door locks before the guy got down from the attick ato unlock his door. So when I was driving long ut of Vegas last year and there was wind storm coming up behind me, driving 80 and the lid on my truck was getting violently blown open from the rear wind, and I didnt want to lose my stuff or have the lid broken off, I had to stop to see if I could fix it and there was nothing I could do and realized all i could do was put something heavy over my stuff so it couldnt blow out and then felt stuck because I had to keep on arm on the lid the whole time while moving 40 pound salt bags with one arm up and over other items in the bed. I concluded that the only soltion was to one, buy new locks to keep it closed and two, drive as fast as I dare to keep the lid from blowing off in teh short term. I got adams help to install the locks and we argued for a half hour about using locks and he began to tell his negighbore lock picking story again and I told him I knew, and explained the wind thing that happened and because hes smart, he pointe dout thats like a one in a thousand scenario. And I said, greta, so going forward when it happens again that one time, I will have lost a mere fifteen dollars on locks rather than hundreds becasue the lid bloew off. And he kept arguing and when all of my points overrode his, he stopped arguing. He even said that if the point is to stop the lid from getting blown off, why have the locks set so the lid stays all the way down, Im already wasting money and time on this and I said, well I will also have the ebnefit of locking if I want and he mentioned his point again, yes but adam, not everyone knows there s alock picking kit every 20 feet on this planet and how to use it. ignorance and the immediate block from robbing me stops a lot of crimes from taking place, let me just have that advantage. most potential robberies are stopped merely by one instance of protection, so a front door with a lock is one insatnce, then getting caught by the police is another, and then the resident having a gun is yet another. But most robberies are stopped becasue someone would do something on a whim and cant because there was a lock in the way or someone near them might see them steal the item. Let me have the one insatnce and all I lose is 15 bucks to protect me against most scenarios. I seriously dont consider myself a brilliant person but seriously, smart people say these things and I know what it is, they have seemingly great ideas really quickly because they have a high IQ, I dont necessarily, my reasoning comes from really pondering over long periods of time, and I have found that with time, I usually can out argue most points high IQ people make who are not willing to take the time that I musttake. My ideas are stupid in the short term, but get really fleshed out, whereas theirs are instantaneous and look really sharp. But its like a pencil thats sharp compared to all the other pencils but thats because it hasnt been used for any length of time. Not to say quick and sharp isnt increadibly useful, but its also not the only thing that matters and not all quick and sharp solutions stick. I also told adam I wanted the lid as low as it goes with teh locks so that wind cant slip inder the lid and blow it up, he saida thats a small gap and I said, yeah but at 80 miles an hour, thats all wind needs to rip off my locks and now my lid too. For the first time ever, I whitnessed a smarter person than me say "Oh, that'sd a good point." I had time to think about things like this so I was prepared. So for once I was ready with a great argument that couldnt be disregarded. I was about to say blown off but...I'm not a huge pun guy eveb if occasionally...
    • Bo then attacked using windows server and then said why not just use win10 and I was like, yeah but now I need a system that can run win10, it wont be low power or low heat or low footprint on the processor or ram. And now I have to still buy something for it. After my experience arguing with adam I was like, hey Bo, look, even if its a bad idea, I see where youre going, but this is not only a solution I would really like to use, its also a great learning oppertunity, which I dont want to spend much money on. He was trying to tear down those points already before I finished speaking and I said, oh and when I write about this in my blog, its going to help me get a job, and then he lost a lot of steam there and said, yeah, I see your point. Yeah thats good. And then I went on, later after he had continued to tear my project apart, and I said yeah I see your point, however, I need to be alowed to make my own mistakes no matter how unwise you think it is to do this. Then he went on. I really thought I lowered the boom on him but no. He skipped a beat and then continued. And then he attacked the very essence of even having a file server at all. why not just use google remote desktop. its like google drive but you install it on all computers incvolved and then you can retrieve any file you want from any computer at any time. (I didnt say this but, yeah well this means I need to leave my desktop running 24/7. I dont want to do that. My computer actually really heats my room up. Id like to have a break from that. Yes, I know, the server would also heat my room, up, but you see, its so low performance, and its just running Linux, so it wont heat my room up nearly as much. We're talking about solutions like one person owning two cars because one is huge and goes right through the gas and is great for special use cases that its used for all the time, and then owning a sall sedan that barely uses the gas by comparrison and you can do most of your regular driving in that. Sometimes these things just make sense. If you only have money for one car, okay fine. Thats not the situatiob im dealing with here, used compuefrs running linux are cheap. And then Bo went on, in fact, you really should just store everuythign on the cloud. I told him my same argument I gave him 4 years ago, what if I come up against a situation where I have to explain my expenses because I need finaincial help. Like relatives for example, they will only help you if youre doing everything you can to help yourself first. I dont want to have to explain a cloud service expense. I buy the parts one time and it mine. Four years ago I said this and he recoiled immediately and said, well, yeah thats a good point I would see why you would do that. You buy the parts one time and they are yours and you dont have to keep spending money on them for eons. its like having a stroage unit, after enough time, that storage unit costs more to keep over the long run than what the worth of the items within. I would never have to explain the power consumption costs to anyone hlping me with bills for example. it could have been my AC running just as easily as my computer and not many people are going to scrutinize a power bill much unless its just really exorbitant. But lets be honest here, computers dont require that much power over the corse of a month, especially not the one I use. My portable AC easily uses more power and it barely raises the price by ten or fifteen dollars a month. 
  • I concluded that after talking to brother landsem, okay look, is SRAID more reliable than HRAID and he said yes absolutely no question. Hes had all raids fail on him but he was always able to recover hardware raids and he had a dssoftware raid he wqas simply unable to recover. then he told a story about sitting with a microsoft guy and he showed him how great SRAID was and Bo was like, and to recover it? And the guy saud, oh, well theres no way to recover it, but other than that its amazing. And i told him that doug said you can recover it, with both SRAID and HRAID, you simply back up the 1KB configuration file that goes with the RAID. And then Bo made the point that maybe he just doesnt know, its been a bit, which are his words, since he talked to this microsoft guy and maybe things are different now. I thought, yep, hes probably saying 20 years since that conversation, and doug is absolutely all over me about how SRAID is recoverable and he uses then all the time and I want to know how reliable vs HRAID and he says hes never had an SRAID fail on him. Bo questioned, okay, well is he just your instructor or does he really know what hes talking about. I feel like qauetions such as these really question my intelligence. Im being really thoghtful about this, is there any part of this that demonstrates me to be simple minded on the matter? I said, okay, Doug is not only an instructor at MTEC at night, but he is a system admin at Brigam Young Universtiy full time and also does admin work for some other company too. Oh, yeah well he probably knows what hes talking about. but i have had almos every raid fail on me or something like that, he said soemthing just like that I just cant remember the exact wording. and owen the MTEC security guard whos also in our class, he uses software raid too and it hasnt failed him. I asked hi how long its been implemented and he said soenthing like nine months and I was like, oh, and kept it to myself that that's not exactly a reliable example when I was talking to Owen. 
    • Since then I have concluded, hold on, Bo isnt considering privacy issues, the fact that prices for cloud services can rise at any time, what if I cant make the paymet that month, do I juts lose all my stuff? I want for the only person that has access to my stuff to be me and those I specifially give access to, not some company that can access it and says they dont just for some revelation in the news that they do spy on your stuff...oh but theyre not specifically pilfering your fiules, they just need to have access for diagnotic reaosns or to monitor your usage hapbits or something. Uh, no. I am satisfied with my usage habits and dont need some company deciding for me anything. Totally, this is what I should have said. I am not paying one cent I dont have to pay. I have religiously used Dropbox for ten years, at first you could get 3GB for free and you were incentivised to connect as many devices to it as possible and they would give you more space for free. I did that. They didnt give me as much space for all the devices I connected as they promised but a little over 5GB has been very useful. But its not anything like what I would do if there were o limits, like there would be if it was my server, my hard drives, my cloud service that I buy one time and use until it breaks or outlives its usefulness. and then about a year ago now, dropbox up and decided that you have to pay more for any features, and they limited the amount of devices you could access it with to three if you wanted to continue to use the free version. You know something thats great about owning my own truck, no one gets to control how many tools I am allowed to stroe in it. If I want to store three hammers for no apparent reason, I can do it if I want. That is freedom. My freedom is being limited because dropbox isnt having as much success making money and so they are tightening your purse strings in order to get the same servies out of you. Oh yeah, its only five bucks, or ten, or twenty. But you know, most regular things now cost at least five bucks at the grocery store things that used to be less than ten are now fifteen or more and so its remarkable how little you can buy before you are just making a quick run to the store and walk out paying 50 dollars for tonight's dinner for one. Hey, Raman every single night for the rest of your life anybody? I know a guy that does this, and I would never be able to do that. to finish up with Bos argument that the amount of time and energy and money you save if you just spend more money for windows for example over linux, I agree, it makes sense, but not in every single scenario. Guess what, if youre trying to get things done and you simply dont have money or you only have it because youre extreemly careful with it like me, then you can just let me have my raid file server running linux that isnt my desktop running 24/7 and heating my room up to the temperature of the sun. 
  • there is a solutuion to my cognitive dissonance regarding why i should use raid 5 at all if i cannot totally rely on the raid to not fail beccause it does after all have one point of failure. The whole point of multiple drives backing multiple drives up like in raid 10 is because you have multiple drives so you have lots of space and even speed is increased because of striping, but then the whole process is repeated a second time identically across an equal number of drives, like having two raid 0's right next to each other but in the same system, and they mirror each other like in raid 1. But unfortunately, any raid controller is going to represent a single point of failure. now when i first heard about JBOD, i seariously thought this had to be a joke or something becuse its literally Just-a-Bunch-Of-Disks, and I was like, why wouldnt you just forget raid altogether at that point and create a storage space in windows control panel? 
  • my idea is to create a JBOD with 3 of the 4TB disks instead of all four, then buy a 12TB ironwolf to back the JBOD up. It'll sort of be cold backup storage, but itll be one thing that doesnt require a ton of configuration that will give me redundancy. And some people say that ironwolf drives have more reliability that WD blue drives, which is what I am used to using. Well, I have purchased a couple of Seagate drives in the past, one is a 3.5" 2TB Barrcuda drive and the other was a Free Agent 500GB 2.5" external drive, the second drive I ever purchased. I dont knwo what happened to the free agent but the 2TB Barracuda is still in use, and I have had it nearly 15 years. But I dont trust it anymore even though it still works. I have had more problems with WD, specifically my 3TB  3.5" Blue drive six years back. I have had other drives fail me but I dont know if they were WD. This option of doing the JDBOD with three 4TB disks instead of four allows me to get away with buying just a 12TB ironwolf instead of buying the 16TB for much more money and with extra space I wouldnt need to have backed up. It would make sense to do all four 4TB drives and get a 16TB ironwolf to back them up but I dont need that much space and 16TB is way more than I need and it is more expensive. I dont want to this to be taken as scripture, but it seems to me that one drive with even more memory might potentially be more prone to failure than the smaller drive too. Dont quote me on that. I would only refer to ironwolf in this case anyway because it is merely cold staorge or level 1 redundancy and not my main storgae solution. 
  • explain storage spaces control panel)
  • started the computer up and switched from ahci to raid mode, then from legacy bios to uefi mode, and the omputer would only load to the EFI Shell. So it still has the same problem it had before we flashed the bios.
  • this was the point that doug suggested we try installing ubuntu server over legacy bios, which is supposed to be installed on uefi rather than legacy bios. But he recalled that there are versions that do work with legacy bios. I did some research and everything i found suggested things like using Rufus, and then using the MBR partitioning scheme.  I immediately stopped and thought, yeah we're doing SRAID. I cant use MBR because none of my 4TB drives would ever be recognised as more than 2TB. I did research however if there is any way to use MBR but not have it affect your storage, and basically what it cmae down to is that if you use legacy bios, youre using MBR, if you use UEFI, then you have the option of using either MBR or GPT. So I dont see any way around using a SRAID. 
  • Talked to my instructor about my JBOD plus an Iron Wolf drive. He kept trying to explain to me that I still have no redundancy. I was confused because I'm like, no i have reundancy because of the ironwolf and hes like but if its apart of the JBOD and I stopped him and said no, theres the JBOD and then theres the ironwolf just using something similar to windows backup to constantly backup the JBOD. Then he explained again that I still dont have redundancy in a way I didnt understand. He then said that the raid itself would have no redundancy and I said, yeah I know. My problem is with RAID itself having a single point of failure because the raid can fail even if the drives dont. Then he said yeah but you have the config file that can rebuild the RAID. And I said I dont trust that. Thats why I am going with the ironwolf. Long story short, he said something that I cant repeat because I was tired and couldnt think when he said it, but somehow he convinced me to go back to using the RAID 5. And then he said that what we can do is after we create the raid, we can then break it on purpose after i transfer all my data to it, and try to bring it back so i can see how it works and whether or not itd work. And this will be okay because all my stuff would still be on the old drives, i wouldnt delete them until after we run this test. 
  • I explained the problem with BIOS and MBR being tied at the hip to doug and he said something interresting. "Yeah, but why when you switch to legacy bios mode on the new motherboard, why then does the board recogize all 4TB on all four drives? I asked if he recalled the old motherboard recognized the drives being 4TB each and he said no. I thought, well, that is a greta questuon. This is where he suggested we try to update the raid controller itself rather than just the BIOS. So now I'm looking for an AMD SB950 driver to then flash the motherboard RAID controller with. I asked Nate earlier about this and he said there was nothing. Oh well. Here we go. I have a feeling this wont solve it either, haha. I am totally willing to do SRAID at this point because Im going to also buy an IronWolf to back the RAID up, so this really is just achademic at this point. Cracking my knuckles, here we go. 
  • in the process of looking for this, i made a discovery that was immediately crucial for SRAID, so we just sort of forgot about what I was doing because I found a website where someone was having similar issues to me except they were doing an SRAID, and while reading it they said that they couldn't get the SRAID to work and another user commented saying that they needed to install ubuntu server on a non-raid drive in order to make it work. 
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!on unix.stackexchange "

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Why is uEFI firmware unable to access a software RAID 1 /boot/efi partition?
Ask Question


BIOS firmware can boot a BIOS formatted /boot partition installed on a software RAID 1 pair no problem. It can even boot from a /boot installed on LVM volume that lives on a software RAID 1 pair.

But with a uEFI install, /boot/efi has to be on a non md partition or the firmware can not access it.

Is this a flaw with uEFI firmware? Or is the problem with how Ubuntu sets up /boot/efi on software RAID devices? Could it be a flaw with how GPT partition tables present software RAID to the firmware?

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!For reference, I’m using: Ubuntu Server 14.04.3 64 bit mdadm RAID setup from the 'Manual' option in the partitioner.

So we immediately went to strategizing how we were going to overcome this hurtle because since the beginning Doug has been saying that we'll just install the OS on the RAID because you can do that in Linux, but not with Windows. However, the problems with the UEFI firmware has caused this need for another drive for the OS to re-emerge.
So I'm going to buy a 2490GB Kingston drive for 22 bucks as soon as I get paid, was going to do 125GB but apparently that ones 23 bucks, so...okay, Ill spend less and get more, thanks Kingston.
I want to get going on the project and was going to grab a magnetic laptop drive from home that was out of use and I had three, one of them is being used to back up all my stuff from my 4TB drive I decided to use for this RAID, so Id rather not add that and risk losing the data to a random drive reformat. The other two unused laptop drives turned out to both be out of commission completely. So I clered off a jump drive I use for situations like these and am using that for the OS until I get the kingston drive, and somehow we're going to mirrir the data or something.
Came back to class tuesday night, and here we go. Then I spent the whole class being stuck by the boot utility. Since ate installed win10 on one of the drives meant for the RAID, it kept trying to boot into that the second I swtched out of SATA RAID mode and back into AHCI mode. I kept turning all the other boot options back to disabled, and even though I pressed f10 to save and exit immediately, it wasnt recognizing the boot order settings. It recognized that I switched to AHCI, so I dont know what was going on there. I did it a second time and then escaped and formally selected save and reboot, and it stuck that time. I was still stuck on the boot problem for some reason but then we spammed all of the potential keys, we dont know which one allowed us to access One-Time Boot Menu but with this board, the MSI 970a-G46, we spammed F10, F11, and F12, and it worked. Once we got there we selected the second USB that had the Ubuntu Server OS files on it.
I ended up sharing this second USB drive with Jack in my class because he and his group last thgrusday were doing a sort of similar project that they were having terrific difficulties with and were not able to solve, and even Doug looked at it and couldnt figure it out so both yesterday night and tonight Jack put the computer back together to try and create a dule boot with win10 and ubunutu server. They were having a lot of similar problems to us. They could install windows but not linux, or this happened or that, I think he ended up reseating the RAM at one point but he also took off the heatsync and re-pasted it too just in case. And last ngith he got ubuntu server installed and then installed windows or the other way around and the installation fo one corrupted the other. Tonight, he spent hours troubleshooting this before doug discovered that the hard drive is in MBR partitioning scheme and windows wants to create all these partitions, and MBR will only recognize up to four and with both linux and windows installed, it wanted five or six, I dont remember. So doug told him to reformat for gpt and he thought he did. But he installed both windows and linux, they both worked, he could switch betweent eh two and discovered from windows that the dosk was still in mbr. So, what gives? He spent about forty five minutes trying to figure out what the deal was with that even though it was working before doug came around and said that actually what happened is that it actually is in mbr but the reason why its working is that he created three partitions for windows 10 to install on and tricked it into just putting those partitions together even though the system still saw them as seperate partitions, and then when he did linux, it created another partition for linux and then a left over space was left blank and unable to be accessed because MBR only recognizes so much disk space. However, some of it could still be recovered and added to the linux partition from withi linux or else if you did it in windows, the linux partition would go nuts.
SO I used a USB that belonged to a friend and put ubuntu server on that so jack and I didnt have to share
We started loading ubuntu server, selected the language, the keyboard layout, skipped the network setup, I only stopped at the page for Guided Storage Configuration to use entire blank USB disk I cleared my stuff off of until I get the kingston drive, and then changed the disk to have ubuntu server installed on from one of the raid drives to my USB since after all I want that drive for the raid. ;I dont want to lose a whole 4TB to a 500MB OS. Then i turned LVM group with LUKS off because im not doing any virtual machine stuff with this system. And then at custom storage, turned it on with the space bar.
On this next page, Storage configuration, one of my 4TB drives needed to be reformatted for the RAID to work, and so we selected that drive and a small drop menu had the reformat feature. You have to rpess okay at the bottom of the screen before it officiallly reformats. We also switched my USB drive from exfat to ext4, we selected RAID 5. Under Format and/or mount md0, which is how we tell it where we're mounting the RAID, in Linux you have to tell it where to mount the drive because windows does C drive and D drive and so on, but you have to specify that in linux.
we changed the boot order back to UEFI ubuntu which now shows up since its been installed, as well as USB  install.

We skipped all the defaults so it could install really quick.

On Wednesday after plugging in at home, connect to network log in and then do !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"ip a s" does the same as ipconfig, see if there's an ip address, can write it down so we can SSH into it tomorrow. leave server running

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Fix explanation of RAID 5 on parity, explain how parity works and get rid of compression

Fun point to learn, parity uses checksum. what it does is it puts all the data in columns of 0s and 1s and however many total bits you have, there's a last column at the end that counts as one more bit, but what this last bit is representing isnt actual data, but a key that is used to resurrect lost data or to check data that could be in error. So this checksum uses this last bit on the end to represent even or odd numbers of bits. If you count a whole row of bits across all the columns, and you add the them up, is that number even or odd? If its odd, then you write a one, if its even then you write a zero for that last bit, so if you lose one for example, by losing a whole drive in your raid array, it can use all the data across the drives plus the parity bit at the end that represents a whole bunch of data for each bit and it can do math to basically add up what the number for the data would have to be in order for the parity bit to make sense in the resurrection of the lost variables or data on the bad drive, or in the case of ECC memory, there's a 50% chance that the EEC checksum bit will catch the errored data and be able to correct it and that 50% chance for catching errors is way better than 0% chance.

!!!!!!!!!!!!!!!!!!!!!!!!!1go back to raid 10 and explain nested raids???

FYI JBOD offers way more features than storage spaces does in Windows control panel, because you can incorporate either striping or mirroring or both>need to look that up to make sure.
and there are raid 50's, 60's, even 100's. i think that's something like a raid 5 with a total raid 5 stripe-i think, or raid 6 with a mirror, or a 10 with an extra mirror-again, i think, I'm probably wrong somehow. And there's a 51 raid too, a raid 5 with a mirror attached I think. actually nesting introduces tiers of raids. so a raid 100 means there's three tiers because each digit of a raid number means that there's two raids on top of each other or more. so raid 10 is a raid 0 stripe of two raid 1's, and a raid 100 is literally a raid 0 that's divided up into two, which is divided up into two more each, then of a raid 1.

Now on Wednesday night, I left my desktop running that had all the files saved on various drives connected to it, and I left it with Anydesk running with an Unattended Access password st up on it so I can activate the remote connection from one side, and I left the server running and connected to my router via ethernet.
I'm in class now and we don't have much time left, we were going to do a project day today because there's a vacation thing at the end of this week, so we don't have to come in Thursday this week. But one of us was struggling understanding RAID and so our class went over it for anyone interested and I sort of got sucked into it even though I already knew most of what he taught us today. So now I'm spending the last 50 minutes of class doing my project, starting with SSH-ing into my file server that's at home.
(Don't know if I mentioned that i installed the minimized version of of ubuntu server, which only does command line so there's no desktop or GUI. its just a UI, no graphic to go with the user interface.
I SSH'd into the server through my VPN, but as I used PowerShell before to do the Pi Hole Pi VPN project, I thought id just open PowerShell. Nope. There's like five different versions apparently, I selected PowerShell x86, which is an older version of PowerShell, and does not recognize the SSH command. And there's also PowerShell ISE and SIE x86. ISE is for writing scripts apparently.
I'm connected now, and I used Anydesk to connect to my desktop so I could at least start backing up my data from drives I don't trust very much.
Doug explained that I don't have a file server yet. Right now I just have a server OS with a RAID. I need to set up the file server before I can set up its connection to other computers on a network.
I selected the file containing all my backup data, right clicked, went to properties, sharing tab...but then Doug stopped me and said there was a better way. We then tried to install SSH on my desktop at home from class over my laptop through the VPN. This was unsuccessful.
Then we tried transferring files from the desktop to the server by_________________, which seemed to work for a little bit until Linux stopped us because it didn't like the way I had named many of my folders, which Windows was fine with. If I had to guess I would say that it might have had a problem with special characters I might have used like underscores and dashes but not necessarily those.
I discovered I was unable to access my router though my VPN. as long as I'm in my Wi-Fi network there's no problem but through the VPN is no good. it will have to be repaired but for now i can remote into my desktop anytime i want as long as its running and Anydesk wasn't closed and I can then access my desktop remotely and then my router from there.
I got my IronWolf in the mail, as well as my Kingston SSD for the Ubuntu Server OS and for installing Samba which will turn my server into a file server which can serve files to network clients.
I attached the hard drives to my desktop, and then (picture case and talk about my concerns before buying it), and cable managed and then since I was concerned about my router access and other things???????, I brought the server into class yesterday Wednesday 2023-10-25. Jack worked with me. He's probably the smartest one in the class. He wanted to understand what I was doing too so he was interested. Since I had trouble remembering the username and password we have Ubuntu server on the jump drive it was installed on, we decided to just go ahead and reinstall it from scratch on the Kingston SSD. I always write down usernames and passwords for this very reason, but we did something like user and password for the credentials. And then I forgot what it was. And we were in a hurry because class was already over when we configured this. I came in class and Doug reminded me what he set it to, and it was the same username and password we give student use network devices in class. Whatever, starting over so it doesn't matter. This time I'll write everything down.
I thought for sure starting over meant we'd have to remake the RAID 5 too but to my surprise and to Doug's delight so he could shove it in my face that Software RAIDS really are reliable, we had wiped everything and even plugged the drives back in to different SATA ports and the system still recognized the RAID and so we didn't have to remake it at all. We simple continued passed creating the RAID and selected it for use on my server.
We reinstalled Ubuntu server, I updated it before installed Samba just in case, we installed Samba and there was a bunch of configuration involved.
Installing Samba is a file server software that makes any server into a file server, and it sort of fetches files and serves them to clients.
I went home and went into my router settings and gave my server a static IP address in my router settings. Then on both my desktop and laptop, I went to a File Explorer folder and right-clicked on This PC, and selected Map to Network Drive. This brings up a window to select the drive letter that you want your computer to identify this drive as, then you type "\\" and IP address of the file server, then another backslash and the folder name you want to map directly to at that IP address, no spaces. reconnect on sign in will already be selected. But make sure to select to connect using different credentials. This makes it so whenever you try to access this file server, windows doesnt try to log in using your windows credentials. Press finish and then a window for entering your network credentials for the server will appear. This is the login for Samba. Its important to keep the login for the server and the login for samba seperated. When you want to mess with the server itself, such as installing samba for exampl,e, you want the server credentials. If you want to access the files, then you want the credentials for samba that you created when you install samba. I was confused by this until just now. I wrote these credentials down and didnt know which scenario I would use either so this is what theyre for. After you input the credentials for mapping the network drive, a new folder will open for the file server whcih you can add stuff to. Start transferring files.
Now i had some particulars I wanted for my server. I am sharing it with my mom and I often want to send her stuff and absolutely hate having to load things on jump drives and go over there and make sure I grab my jump drive before I leave and all that. I can send it and she gets it when she wants to use it. And thats it. Its like using Venmo instead of going to the store or bank and taking out cash just to pay a friend back for something. So much less trouble. So I also created a user account for my mom, but her user doesnt have access to any of my stuff because she often fumbles with what she does on the computer and I dont want all my photography randomely deleted one day and not even she knows how it happened. However I will only give her access to her folder and I gave myself access to both her folder and mine, so its a simple matter to transfer stuff back and forth.
I also mapped her folder to both my desktop and laptop and then on my iPhone I simply went to the Files app, and under the browse tab, I selected the botton at the top right of the screen, which displays a circle with three dots in it. then I selected the connect to server option, I didnt have to worry about backslashes here, which is a good thing because IOS simply doesnt provide them here anyway. So I typed in the server IP address, on the next page it simply showed both mine and my moms folders. So now I can move everything off dropbox and just use my server.
this isnt done quite yet. Now I need to set the server up to automatically back the raid up to my 12TB IronWolf. Doug wanted me to look up "rsync" with "cron".
I went to start studying it and thought i should SSH into my server to get started, and I actually ran into this problem with the Pi Hole and I completely forgot about it so I dont know if I write about it but I am going to write it here for good measure. Its an SSH fingerprint mismatch. What happened is that because we used SSH on my laptop to access the file server once already but it was for the installation that was on the jump drive which is no longer in use, my laptop was expecting to connect to that installation and the fingerprint was different. So it have me the warning "@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed." To fix this, on my laptop I simply went to my C drive, users, clicked on the user, at the top of the folder was a subfolder called ".ssh", I entered that and inside were two files, one was Old Hosts and the other was Known Hosts. You delete that Known Hosts file and then retry the SSH connection and it should work this time.
I tried again and this time it asked me if I wanted to create a fingerprint. I said yes.
Now moving on with RSYNC with CRON. Actually, RSYNC stands for Remote Sync and is not really a backup tool but can be used as part of a solution to for backup. This is the loophole I gather that I'll be using as according to a google search, when you combine it with other tools such as CRON, it can be used for backup. So it's not actually a backup tool, instead it actually transfers files and synchronizes them. And CRON stands for Chronos, derived from Greek referring to time. But in its full expansion, CRON is like Chronicle without the 'h'. Some will say that CRON stands for 'command run on notice', but that's not it. So put together, RSYNC with CRON will give me a backup solution. And so when you hear someone say CRON job, you know they're referring to a command line utility time based job that you can create for various tools including RSYNC. And what's nice about RSYNC is that it won't just transfer over a bunch of files every time there's a change, it'll sync them so any small changes on one side reflected over to the backup side, which requires less bandwidth.
Another   good thing to know is that are different types of backups called GFS or Grandfather backups, Father Backups, or Son Backups. This can seem a little confusing because these backups are--I think, more commonly known by their more direct names. A Grandfather Backup is the same as a Full Backup taken once a month or on some regular basis that isn't everyday, usually more like once a moth. A Father Backup are usually Incremental or Differential Backups that happen in a more common interval such as weekly. Incremental Backups are just making regular changes or additions on a regular basis and are small and frequent while Differential Backups are very similar to Incremental Backups except they are much larger and less frequent, maybe like every week or something by comparison. And Son Backups are Incremental Backups which happen every day, treated like additions or changes to the previous backup whether it was another Incremental Backup or a full Backup. 

In cron we set minute, hour, day of the month, day of the week, and month of the year, and I did wednesday at 3am every week so we typed 0 for the first minute of the hour, space, 3, space, * because we arent setting for one day of the month, we actually want a certain day of every week, however wednesday every week could land on any day of the month in the future so you put asterisk to mean 'everything'. And youll notice that all the text above this command is in blue, that is because theres a hash mark at the beginning of each of those lines which tells the program to ignore those lines. So when you type the minute hour day etc., do not include a hash mark or else cron will ignore it and wonder why you didnt set the min hour etc.
cron opens in a text editor of your choice so when you enter the configurations you want, you have to save them rather than press enter. You save by pressing control + S if editing in the Nano text editor.
also, class is ending so we want to start cron backing things up right now, and if we get the file tranferring going and then I sleep my laptop and leave, my ssh will end and because it ends, the file transfer will end as well. So doug showed me that if you type "screen" and that'll give you another screen where you can carry out tasks in ubuntu started from an ssh that will not end the task when the ssh ends. When you first use screen, press eneter to switch to the new screen, to detatch from that screen press ctr + A and then ctr + D. To then return to it again, do another "screen -r" command and itll bring me back to where it is.
"sudo rsync -aP /mnt/raid /mnt/ironwolfe" starts the manual backup instead of waiting for it to run the command later. Itll do it again at 3am but...whatever. Captial P in command means progress bar within the command line

In a cron job, the day of the week can be represented as either 0-6 or 1-7, depending on the system and the cron implementation. Here's the typical mapping:0 or 7: Sunday
1: Monday
2: Tuesday
3: Wednesday
4: Thursday
5: Friday
6: Saturday

So, both representations are commonly used. You can use either 0-6 or 1-7 to specify the day of the week in your cron job, and it should work as expected on most systems.

I conclude that at the end of this, I had spent a total of nearly $500 on my file server. The DIY case was between $60 and $90, I bought two 4TB hard drives for about $100 or $120ish, the first computer to be used for the project costed $25 but had no power supply, and then when that didn't work I spent between $40 and $50 on a new one, which also didn't have a power supply but also didn't have a CPU fan and might also not have had any RAM either. So Nate added that to his costs for building the computer for me. Then I bought the IronWolf drive to back up the RAID for about $200, or maybe it was a bit less because of a nearly perfectly timed sale.

This has been Truncat3d 00000000111100010100110______________end of line

No comments:

Post a Comment

2025-07-10 - BYU Wi‑Fi captive portal troubleshooting

  BYU Wi‑Fi Captive Portal Troubleshooting What happened I had used BYU Wi‑Fi just fine before, but at some point, the captive portal star...