Switching over to SSD for main storage drive over 10 TB – are we there yet?

[Chester McCheeserton]

Currently using a spinning 12 TB bare 3.5 drive in an OWC enclosure as my master "archive". It has about 75,000 raw files, 2 TB of drum scans, a bunch of 4k video, and 15 years of flatbed and imacon scans. I have an identical spinning drive in a drawer that I back this one up to every month or so, and a 3rd offsite that I try to back up to at least a few times a year. The delay in using lightroom or when trying to use the finder to search for a specific file is getting old, though I can do what I need to do so far. I have a hopped up 2020 macbook pro with 32 gigs of ram and a fancy graphics card, and over 850 gb of free space on the internal ssd drive, I want to get at least 3 more years from this machine.

I don't understand the difference between PCIe and NVMe. nor what would make the most sense. My intuition tells me I should pony up for a single ssd in the neighborhood of 12-16 tb and use that as my primary archive, and continue to back up to the 2 12 spinning drives. What are my options? How much riskier is it to have 2 8tb ssds or 4 4 tb ssds to get the same storage?

I think what I'm seeing as the most cost effective option is getting 2 samsung 870 QVO 2.5 8 tb drives. Can someone suggest a compact enclosure that they could both live in? or a single ssd that would get me similar speed and storage space?

[Alan Klein]

B&H Photo sells photo computers for pros. Maybe they can tell you what's best for the size of data you need to handle.

[r.e.]

I don't understand the difference between PCIe and NVMe.

An NVMe solid state module is mounted in a PCIe slot that in turn talks to your computer's central processor via what sounds, in your case, like a USB 3.1 Gen 2 (up to 10Gbps) or Thunderbolt 3 (up to 40Gbps) port.

It would be helpful to know more about your computer and about what kind of access you need to this data and how quickly it grows. Some reading would help you make a short list. Maybe look into Network Attached Storage (NAS) while you're at it.

You've already dealt with OWC. I've found OWC helpful on the phone, and they can certainly give you some options. As Allan says, so can B&H. Or Adorama.

[bdkphoto]

Currently using a spinning 12 TB bare 3.5 drive in an OWC enclosure as my master "archive". It has about 75,000 raw files, 2 TB of drum scans, a bunch of 4k video, and 15 years of flatbed and imacon scans. I have an identical spinning drive in a drawer that I back this one up to every month or so, and a 3rd offsite that I try to back up to at least a few times a year. The delay in using lightroom or when trying to use the finder to search for a specific file is getting old, though I can do what I need to do so far. I have a hopped up 2020 macbook pro with 32 gigs of ram and a fancy graphics card, and over 850 gb of free space on the internal ssd drive, I want to get at least 3 more years from this machine.

I don't understand the difference between PCIe and NVMe. nor what would make the most sense. My intuition tells me I should pony up for a single ssd in the neighborhood of 12-16 tb and use that as my primary archive, and continue to back up to the 2 12 spinning drives. What are my options? How much riskier is it to have 2 8tb ssds or 4 4 tb ssds to get the same storage?

I think what I'm seeing as the most cost effective option is getting 2 samsung 870 QVO 2.5 8 tb drives. Can someone suggest a compact enclosure that they could both live in? or a single ssd that would get me similar speed and storage space?

I use a 4TB NVMe (2 2tb NVMe ssd in a Raid O Netstor enclosure ) as my working drive (Thunderbolt 4 speeds) this is backed up nightly to a 4 TB spin up drive. As the drive is filled up I back that drive up - and back up the raid ) and erase to reuse so I have a raid 0 ssd working disk, and my archives are on spin ups. This make a cost effective and fast working drive and the spin ups are used as back ups, and archives (I prefer to use smaller drives and more of them rather than the big platters). All this keeps the current files on a super fast ssd drive for working files, and spin up is used for archive/backup. If I need to work on the older files from the archive either I live with the slower connection (usb 3) of I can move the files to the ssd and work on them there - so far I have not needed to to move them as the files load easily from the spin up and work just fine. (mac M1 mini)

[Chester McCheeserton]

An NVMe solid state module is mounted in a PCIe slot that in turn talks to your computer's central processor via what sounds, in your case, like a USB 3.1 Gen 2 (up to 10Gbps) or Thunderbolt 3 (up to 40Gbps) port.

It would be helpful to know more about your computer and about what kind of access you need to this data and how quickly it grows. Some reading would help you make a short list. Maybe look into Network Attached Storage (NAS) while you're at it.

You've already dealt with OWC. I've found OWC helpful on the phone, and they can certainly give you some options. As Allan says, so can B&H. Or Adorama.

Yeah I have 4 thunderbolt 3 ports. My computer is actually a late 2019 16 inch mbp. I need access 2-3 times a week to the data. It grows on overage 1 gig per week, but then sometimes if I'm on a trip or something that # increases a lot.

I'm asking here because I think the people who work at OWC and B&H are kind of like car salesman rather than race car drivers. No disrespect intended to car salesman or computer store workers.
I just thought there might be folks on here who had dealt with something similar, for themselves and their own work.

If I get two 8TB NVMe drives and run them in an external enclosure, will my read and write speeds still be dramatically faster than running 2 8 tb SATA 111 drives in an enclosure considering both are having to run through the thunderbolt 3 wires?

[Chester McCheeserton]

I use a 4TB NVMe (2 2tb NVMe ssd in a Raid O Netstor enclosure ) as my working drive (Thunderbolt 4 speeds) this is backed up nightly to a 4 TB spin up drive. As the drive is filled up I back that drive up - and back up the raid ) and erase to reuse so I have a raid 0 ssd working disk, and my archives are on spin ups. This make a cost effective and fast working drive and the spin ups are used as back ups, and archives (I prefer to use smaller drives and more of them rather than the big platters). All this keeps the current files on a super fast ssd drive for working files, and spin up is used for archive/backup. If I need to work on the older files from the archive either I live with the slower connection (usb 3) of I can move the files to the ssd and work on them there - so far I have not needed to to move them as the files load easily from the spin up and work just fine. (mac M1 mini)

Thanks – Yes I can see that running Raid 0 would need a regular and consistent back up...

[r.e.]

If I get two 8TB NVMe drives and run them in an external enclosure, will my read and write speeds still be dramatically faster than running 2 8 tb SATA 111 drives in an enclosure considering both are having to run through the thunderbolt 3 wires?

8TB of MVMe M.2 solid state drives will be much faster than the Samsung SATA drives that you're talking about, but also a lot more expensive. The reason is that you'll get Thunderbolt 3 speed with the NVMe drives but not with the SATA drives. The latter are limited to USB 3.1 Gen. 2 speed (10Gbps) regardless of what port you connect them to. Samsung says that the SATA drives have "sequential read and write speeds of up to 560 and 530 Mb/s respectively". Note "up to". Those numbers are optimistic, and they are much slower than what an NVMe M.2 drive can deliver, which via a Thunderbolt 3 port can be over 2500Gbps. If you're interested in RAID 0, that would increase the speed of both types of drive, but as you know with a reliability issue that requires close attention to backups.

[Chester McCheeserton]

8TB of MVMe M.2 solid state drives will be much faster than the Samsung SATA drives that you're talking about, but also a lot more expensive. The reason is that you'll get Thunderbolt 3 speed with the NVMe drives but not with the SATA drives. The latter are limited to USB 3.1 Gen. 2 speed (10Gbps) regardless of what port you connect them to. Samsung says that the SATA drives have "sequential read and write speeds of up to 560 and 530 Mb/s respectively". Note "up to". Those numbers are optimistic, and they are much slower than what an NVMe M.2 drive can deliver, which via a Thunderbolt 3 port can be over 2500Gbps. If you're interested in RAID 0, that would increase the speed of both types of drive, but as you know with a reliability issue that requires close attention to backups.

Thank you. Yes it's def a lot more expensive but I remember looking into this just two years ago when those thunderblade owc things came out, they were like 4 grand for the size storage I'd need. I think doing it this way gets more in the ballpark of 2500, which is still insanely expensive, But starting to seem more within range.

[r.e.]

Thank you. Yes it's def a lot more expensive but I remember looking into this just two years ago when those thunderblade owc things came out, they were like 4 grand for the size storage I'd need. I think doing it this way gets more in the ballpark of 2500, which is still insanely expensive, But starting to seem more within range.

If you're prepared to spend that kind of money, you'll need an M.2 NVMe drive enclosure and a PCIe M.2 carrier. Several companies make one or both: OWC, Sonnet, and High Point, to name three, make both. There's an interesting Canadian company called Amfeltec that makes carriers, but not enclosures. It makes these, called the Squid, for 1, 2, 4 and 6 M.2 drives. Your MacBook doesn't do something called bifurcation, which you should read about before going further with this. One of the attractions of Amfeltec's Squid carriers is that they incorporate bifurcation in the carrier itself. If I recall, this is discussed in the video below, along with the difference between 16 lanes and 8 lanes.

Normally, people install a carrier directly in a PCIe slot in, for example, a Mac Pro tower computer. As you know, to use these as external drives for something like a MacBook, one uses an external enclosure. This video is about installing one of Amfeltec's carriers in a 2019 Mac Pro and how it compares to OWC's product. I think that the video also does a good job of explaining the RAID options. The title says Part 1, but I don't think that there is a Part II:


https://www.youtube.com/watch?v=4tiZR5THMV4

[r.e.]

Further to the above post, this is a really good explanation, from the series Explaining Computers, about PCIe:


https://www.youtube.com/watch?v=PrXwe21biJo

[Chester McCheeserton]

Your MacBook doesn't do something called bifurcation, which you should read about before going further with this.
This video is about installing one of Amfeltec's carriers in a 2019 Mac Pro and how it compares to OWC's product. I think that the video also does a good job of explaining the RAID options. The title says Part 1, but I don't think that there is a Part II:



Thanks. I am not familiar with bifurcation but I think maybe it sounds like it might be less of an issue running just 2 ssds rather than 4 or 8...I don't really need insane speed I'm not doing a ton of 4k editing, mainly want LR to not freeze up and to have to stop and read a newspaper every time I use finder to locate a specific file name. seeing them use disc utility in the video to set up a RAID is helpful.

this would work right? or is it missing the PCIe carrier you mention...
https://www.amazon.com/Sabrent-Thunderbolt-Tool-Free-Enclosure-EC-T3DN/dp/B08S5JPWR6/ref=cm_cr_arp_d_product_top?ie=UTF8

[MAubrey]

If you're thinking about replacing spinning disks with ssd's for long term storage, you should consider how bit rot affects ssd's as compared to magnetic hard drives.
https://www.howtogeek.com/660727/bit-rot-how-hard-drives-and-ssds-die-over-time/

[Alan Klein]

Maybe you can check with a local wedding photographer who does a lot of business.