Page 1 of 2 12 LastLast
Results 1 to 10 of 13

Thread: Switching over to SSD for main storage drive over 10 TB are we there yet?

  1. #1

    Switching over to SSD for main storage drive over 10 TB are we there yet?

    Currently using a spinning 12 TB bare 3.5 drive in an OWC enclosure as my master "archive". It has about 75,000 raw files, 2 TB of drum scans, a bunch of 4k video, and 15 years of flatbed and imacon scans. I have an identical spinning drive in a drawer that I back this one up to every month or so, and a 3rd offsite that I try to back up to at least a few times a year. The delay in using lightroom or when trying to use the finder to search for a specific file is getting old, though I can do what I need to do so far. I have a hopped up 2020 macbook pro with 32 gigs of ram and a fancy graphics card, and over 850 gb of free space on the internal ssd drive, I want to get at least 3 more years from this machine.

    I don't understand the difference between PCIe and NVMe. nor what would make the most sense. My intuition tells me I should pony up for a single ssd in the neighborhood of 12-16 tb and use that as my primary archive, and continue to back up to the 2 12 spinning drives. What are my options? How much riskier is it to have 2 8tb ssds or 4 4 tb ssds to get the same storage?

    I think what I'm seeing as the most cost effective option is getting 2 samsung 870 QVO 2.5 8 tb drives. Can someone suggest a compact enclosure that they could both live in? or a single ssd that would get me similar speed and storage space?

  2. #2
    Alan Klein's Avatar
    Join Date
    Jun 2015
    Location
    New Jersey was NYC
    Posts
    2,010

    Re: Switching over to SSD for main storage drive over 10 TB are we there yet?

    B&H Photo sells photo computers for pros. Maybe they can tell you what's best for the size of data you need to handle.

  3. #3

    Join Date
    Jan 2007
    Location
    New York
    Posts
    2,575

    Re: Switching over to SSD for main storage drive over 10 TB are we there yet?

    Quote Originally Posted by Chester McCheeserton View Post

    I don't understand the difference between PCIe and NVMe.
    An NVMe solid state module is mounted in a PCIe slot that in turn talks to your computer's central processor via what sounds, in your case, like a USB 3.1 Gen 2 (up to 10Gbps) or Thunderbolt 3 (up to 40Gbps) port.

    It would be helpful to know more about your computer and about what kind of access you need to this data and how quickly it grows. Some reading would help you make a short list. Maybe look into Network Attached Storage (NAS) while you're at it.

    You've already dealt with OWC. I've found OWC helpful on the phone, and they can certainly give you some options. As Allan says, so can B&H. Or Adorama.
    Arca-Swiss 8x10/4x5 | Mamiya 6x7 | Leica 35mm | Blackmagic Ultra HD Video
    Sound Devices audio recorder, Schoeps & DPA mikes
    Mac Studio/Eizo with Capture One, Final Cut, DaVinci Resolve, Logic

  4. #4

    Join Date
    Mar 2022
    Posts
    29

    Re: Switching over to SSD for main storage drive over 10 TB are we there yet?

    Quote Originally Posted by Chester McCheeserton View Post
    Currently using a spinning 12 TB bare 3.5 drive in an OWC enclosure as my master "archive". It has about 75,000 raw files, 2 TB of drum scans, a bunch of 4k video, and 15 years of flatbed and imacon scans. I have an identical spinning drive in a drawer that I back this one up to every month or so, and a 3rd offsite that I try to back up to at least a few times a year. The delay in using lightroom or when trying to use the finder to search for a specific file is getting old, though I can do what I need to do so far. I have a hopped up 2020 macbook pro with 32 gigs of ram and a fancy graphics card, and over 850 gb of free space on the internal ssd drive, I want to get at least 3 more years from this machine.

    I don't understand the difference between PCIe and NVMe. nor what would make the most sense. My intuition tells me I should pony up for a single ssd in the neighborhood of 12-16 tb and use that as my primary archive, and continue to back up to the 2 12 spinning drives. What are my options? How much riskier is it to have 2 8tb ssds or 4 4 tb ssds to get the same storage?

    I think what I'm seeing as the most cost effective option is getting 2 samsung 870 QVO 2.5 8 tb drives. Can someone suggest a compact enclosure that they could both live in? or a single ssd that would get me similar speed and storage space?
    I use a 4TB NVMe (2 2tb NVMe ssd in a Raid O Netstor enclosure ) as my working drive (Thunderbolt 4 speeds) this is backed up nightly to a 4 TB spin up drive. As the drive is filled up I back that drive up - and back up the raid ) and erase to reuse so I have a raid 0 ssd working disk, and my archives are on spin ups. This make a cost effective and fast working drive and the spin ups are used as back ups, and archives (I prefer to use smaller drives and more of them rather than the big platters). All this keeps the current files on a super fast ssd drive for working files, and spin up is used for archive/backup. If I need to work on the older files from the archive either I live with the slower connection (usb 3) of I can move the files to the ssd and work on them there - so far I have not needed to to move them as the files load easily from the spin up and work just fine. (mac M1 mini)
    http://brucekatzphoto.com

    Original join date 2008...

  5. #5

    Re: Switching over to SSD for main storage drive over 10 TB are we there yet?

    Quote Originally Posted by r.e. View Post
    An NVMe solid state module is mounted in a PCIe slot that in turn talks to your computer's central processor via what sounds, in your case, like a USB 3.1 Gen 2 (up to 10Gbps) or Thunderbolt 3 (up to 40Gbps) port.

    It would be helpful to know more about your computer and about what kind of access you need to this data and how quickly it grows. Some reading would help you make a short list. Maybe look into Network Attached Storage (NAS) while you're at it.

    You've already dealt with OWC. I've found OWC helpful on the phone, and they can certainly give you some options. As Allan says, so can B&H. Or Adorama.
    Yeah I have 4 thunderbolt 3 ports. My computer is actually a late 2019 16 inch mbp. I need access 2-3 times a week to the data. It grows on overage 1 gig per week, but then sometimes if I'm on a trip or something that # increases a lot.

    I'm asking here because I think the people who work at OWC and B&H are kind of like car salesman rather than race car drivers. No disrespect intended to car salesman or computer store workers.
    I just thought there might be folks on here who had dealt with something similar, for themselves and their own work.

    If I get two 8TB NVMe drives and run them in an external enclosure, will my read and write speeds still be dramatically faster than running 2 8 tb SATA 111 drives in an enclosure considering both are having to run through the thunderbolt 3 wires?

  6. #6

    Re: Switching over to SSD for main storage drive over 10 TB are we there yet?

    Quote Originally Posted by bdkphoto View Post
    I use a 4TB NVMe (2 2tb NVMe ssd in a Raid O Netstor enclosure ) as my working drive (Thunderbolt 4 speeds) this is backed up nightly to a 4 TB spin up drive. As the drive is filled up I back that drive up - and back up the raid ) and erase to reuse so I have a raid 0 ssd working disk, and my archives are on spin ups. This make a cost effective and fast working drive and the spin ups are used as back ups, and archives (I prefer to use smaller drives and more of them rather than the big platters). All this keeps the current files on a super fast ssd drive for working files, and spin up is used for archive/backup. If I need to work on the older files from the archive either I live with the slower connection (usb 3) of I can move the files to the ssd and work on them there - so far I have not needed to to move them as the files load easily from the spin up and work just fine. (mac M1 mini)
    Thanks – Yes I can see that running Raid 0 would need a regular and consistent back up...
    Last edited by Chester McCheeserton; 27-Mar-2022 at 14:50.

  7. #7

    Join Date
    Jan 2007
    Location
    New York
    Posts
    2,575

    Re: Switching over to SSD for main storage drive over 10 TB are we there yet?

    Quote Originally Posted by Chester McCheeserton View Post
    If I get two 8TB NVMe drives and run them in an external enclosure, will my read and write speeds still be dramatically faster than running 2 8 tb SATA 111 drives in an enclosure considering both are having to run through the thunderbolt 3 wires?
    8TB of MVMe M.2 solid state drives will be much faster than the Samsung SATA drives that you're talking about, but also a lot more expensive. The reason is that you'll get Thunderbolt 3 speed with the NVMe drives but not with the SATA drives. The latter are limited to USB 3.1 Gen. 2 speed (10Gbps) regardless of what port you connect them to. Samsung says that the SATA drives have "sequential read and write speeds of up to 560 and 530 Mb/s respectively". Note "up to". Those numbers are optimistic, and they are much slower than what an NVMe M.2 drive can deliver, which via a Thunderbolt 3 port can be over 2500Gbps. If you're interested in RAID 0, that would increase the speed of both types of drive, but as you know with a reliability issue that requires close attention to backups.
    Arca-Swiss 8x10/4x5 | Mamiya 6x7 | Leica 35mm | Blackmagic Ultra HD Video
    Sound Devices audio recorder, Schoeps & DPA mikes
    Mac Studio/Eizo with Capture One, Final Cut, DaVinci Resolve, Logic

  8. #8

    Re: Switching over to SSD for main storage drive over 10 TB are we there yet?

    Quote Originally Posted by r.e. View Post
    8TB of MVMe M.2 solid state drives will be much faster than the Samsung SATA drives that you're talking about, but also a lot more expensive. The reason is that you'll get Thunderbolt 3 speed with the NVMe drives but not with the SATA drives. The latter are limited to USB 3.1 Gen. 2 speed (10Gbps) regardless of what port you connect them to. Samsung says that the SATA drives have "sequential read and write speeds of up to 560 and 530 Mb/s respectively". Note "up to". Those numbers are optimistic, and they are much slower than what an NVMe M.2 drive can deliver, which via a Thunderbolt 3 port can be over 2500Gbps. If you're interested in RAID 0, that would increase the speed of both types of drive, but as you know with a reliability issue that requires close attention to backups.
    Thank you. Yes it's def a lot more expensive but I remember looking into this just two years ago when those thunderblade owc things came out, they were like 4 grand for the size storage I'd need. I think doing it this way gets more in the ballpark of 2500, which is still insanely expensive, But starting to seem more within range.

  9. #9

    Join Date
    Jan 2007
    Location
    New York
    Posts
    2,575

    Re: Switching over to SSD for main storage drive over 10 TB are we there yet?

    Quote Originally Posted by Chester McCheeserton View Post
    Thank you. Yes it's def a lot more expensive but I remember looking into this just two years ago when those thunderblade owc things came out, they were like 4 grand for the size storage I'd need. I think doing it this way gets more in the ballpark of 2500, which is still insanely expensive, But starting to seem more within range.
    If you're prepared to spend that kind of money, you'll need an M.2 NVMe drive enclosure and a PCIe M.2 carrier. Several companies make one or both: OWC, Sonnet, and High Point, to name three, make both. There's an interesting Canadian company called Amfeltec that makes carriers, but not enclosures. It makes these, called the Squid, for 1, 2, 4 and 6 M.2 drives. Your MacBook doesn't do something called bifurcation, which you should read about before going further with this. One of the attractions of Amfeltec's Squid carriers is that they incorporate bifurcation in the carrier itself. If I recall, this is discussed in the video below, along with the difference between 16 lanes and 8 lanes.

    Normally, people install a carrier directly in a PCIe slot in, for example, a Mac Pro tower computer. As you know, to use these as external drives for something like a MacBook, one uses an external enclosure. This video is about installing one of Amfeltec's carriers in a 2019 Mac Pro and how it compares to OWC's product. I think that the video also does a good job of explaining the RAID options. The title says Part 1, but I don't think that there is a Part II:

    Arca-Swiss 8x10/4x5 | Mamiya 6x7 | Leica 35mm | Blackmagic Ultra HD Video
    Sound Devices audio recorder, Schoeps & DPA mikes
    Mac Studio/Eizo with Capture One, Final Cut, DaVinci Resolve, Logic

  10. #10

    Join Date
    Jan 2007
    Location
    New York
    Posts
    2,575

    Re: Switching over to SSD for main storage drive over 10 TB are we there yet?

    Further to the above post, this is a really good explanation, from the series Explaining Computers, about PCIe:

    Arca-Swiss 8x10/4x5 | Mamiya 6x7 | Leica 35mm | Blackmagic Ultra HD Video
    Sound Devices audio recorder, Schoeps & DPA mikes
    Mac Studio/Eizo with Capture One, Final Cut, DaVinci Resolve, Logic

Similar Threads

  1. Compound shutter main spring?
    By Tim Deming in forum LF DIY (Do It Yourself)
    Replies: 21
    Last Post: 7-Jan-2017, 03:45
  2. Chromega Dichroic main light not working
    By Wade Guisbert in forum Darkroom: Equipment
    Replies: 8
    Last Post: 31-Dec-2009, 18:42
  3. Drobo as main HDDs?
    By Juergen Sattler in forum Digital Processing
    Replies: 9
    Last Post: 21-Oct-2009, 22:40
  4. Can you recommend some Main Streets for me to shoot?
    By Sandy Sorlien in forum Location & Travel
    Replies: 30
    Last Post: 4-Aug-2001, 13:35
  5. old main street photography
    By james norman in forum Style & Technique
    Replies: 2
    Last Post: 16-May-1999, 17:24

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •