r/DataHoarder • • Jun 14 '26

Mod Update Mod update: regarding the flood of HDD/SSD price posts, and AI slop.

945 Upvotes

Regarding HDD price posts:

We empathise with your plight and hope HDD / SSD / RAM prices will come down soon.

However, we don’t need screenshots of high HDD prices to be posted in the sub multiple times a day, everyone already knows that the prices are high.

As such, from this point on the mod team will be removing all posts about high prices, with exception for posts made on Free-Post-Friday’s, or posts which actually have new and meaningful information or discussion.

Regarding AI content and AI projects:

As always AI written posts & comments are not allowed on this subreddit, please report any Ai generated content you see.

The mods have recently been cracking down on the flood of AI generated projects, notably ones of low quality that are nothing new.

If someone has the skill to use GitHub, they’d almost certainly have the skill to ask an AI to code yet another YT-DLP / FFMPEG wrapper themselves.

However, there are useful tools that have been made from AI generated code. If they are something truly useful or new, we do allow them with prior approval.

TL;DR:

  • Mods will be removing [high HDD price] posts, except for meaningful discussion or Fridays.
  • Posting AI generated projects/tools needs mod approval beforehand, and a link to the GitHub repository.
  • Please keep reporting ai-slop.

r/DataHoarder • • 6h ago

Question/Advice scanimage '48-bit color' vs default 'color' scan

18 Upvotes

I'm using scanimage to scan some books. '48-bit color' scan looks too dark and saturated to me, whereas the default 'color' scan looks more accurate to the actual color but a bit too bright. I thought '48-bit color' should look more accurate, why is it doing this? Thoughts and advice?

'48-bit color' scan
Default 'color' scan

I'm using scanimage to scan some books. '48-bit color' scan looks to dark to me, whereas the default 'color' scan looks too bright. Thoughts and advice?


r/DataHoarder • • 22h ago

Question/Advice Moving ~1PB of data

223 Upvotes

What’s the best way to move ~1PB of data these days? Assuming you don’t have local storage to absorb that locally and the window to do it is short, hours to a few days perhaps.


r/DataHoarder • • 5h ago

Hoarder-Setups Anybody have a still working bulk downloader for literotica site?

6 Upvotes

A lot of tools no longer work.

I was using literotica-dl2 which was great. I could point it at an Author page and it would download all chapters for a story into individual .epub files. Weeks later I could do it again and it would skip chapters I already had but it would update stories with new chapters. But it no longer works.

I see "litkeeper", some tapper monkey scripts from a few years ago, etc. Some utilities from "SleezyFork" with odd descriptions.

WHAT I WANT

.epub output, combine all chapters of a story into 1 file. Bonus would be a 'gallery' download where I feed the tool an author page and it finds all their stories and creates 1 epub per story with all the chapters.

Anything you guys use on the regular that still works?


r/DataHoarder • • 5h ago

News ipl.org (the old Internet Public Library domain) goes offline Nov 9, 2026: public pages removed, nothing migrated

3 Upvotes

IPL posted a retirement notice: https://ipl-support.zendesk.com/hc/en-us/articles/43831334972695-Important-Update-IPL-is-Retiring-Everything-You-Need-to-Know

Key lines: - "Effective November 9, 2026, IPL will be officially retired" and "the site will be taken offline permanently." - "We are not migrating any content or product features from IPL to a new platform." - "Public pages on IPL will stop being available after November 9, 2026," and they say cached copies on archive sites are "outside our control." - Accounts and data get deleted; logins end Nov 9.

Context: the domain was the nonprofit Internet Public Library from 1995 to 2015 (https://en.wikipedia.org/wiki/Internet_Public_Library). Since about 2020 it has been a commercial student-essay site, which is what's closing. So the current pages are mostly user essays, but a lot of old links across the web point at ipl.org.

If you've uploaded anything there, the notice says you can download it until Nov 9. Posting in case anyone wants to grab public pages before they go.


r/DataHoarder • • 9h ago

Question/Advice Best practive for entire folder (multiple hundred GBs) on one drive, move to pool? - MergerFS and a dumb question

6 Upvotes

Hiya, I'm running MergerFS and Snapraid for quite some time now but I never ran into this situation.

I have a storage pool (/mnt/storagepool) consisting of /mnt/hdd1, /mnt/hdd2, and /mnt/hdd3 but I had a little config issue with a Docker container which got me into this situation:

I have a folder of multiple hundred GBs that got written directly onto /mnt/hdd2, can I just move the folder with mv or rsync from /mnt/hdd2 to /mnt/storagepool and then just run snapraid sync for good measure? I could also offload that to an external drive and move it back, but that seems unnecessarily complicated.

My NFS shares and all my Docker containers refer or point to /mnt/storagepool and I want this folder to be available through that as well.

Cheers!


r/DataHoarder • • 8h ago

Question/Advice FADGI 3.1 (Bound Volumes: Rare and Special Materials) vs 3.8 (Prints and Photographs) for a bounded photo book?

5 Upvotes

FADGI 3.1 (Bound Volumes: Rare and Special Materials) vs 3.8 (Prints and Photographs) for a bounded photo book?


r/DataHoarder • • 13h ago

Question/Advice I want more books. Anything notable to download (or even purchase bulk) outside of Project Gutenberg?

8 Upvotes

I have a huge amount of storage (48TB usable space, mirrored) and a sweet little personal digital library coming in at about 1.5TB. I’m running some local AI agents on for OCR, classification, identification of curious items / connections between works, and general curation to allow me to pick and choose what I want to read (because let’s be honest, I’ll never read all this without some sort of help identifying what’s good).

But I want more. Legally (or legally grey - not straight illegal) if possible.

I’ve picked up a few humble bundles of ebooks, I’ve downloaded the survival library, I’ve downloaded the internet archive’s magazine scans, I’ve replicated Wikipedia, I’ve downloaded the entirety of the Congressional Research Service’s reports, I’ve downloaded some NASA reports, and I’ve added a few books I was able to pull from my University Library.

I’m looking for reference texts, acclaimed fiction (not hoards of shitty romance novels), literature, interesting research, or otherwise fun things to have around. Likewise, I’m interested in other forms of media - I have a huge collection of museum-quality art that is offered on Wikimedia and museum websites, as well as historical photographs, all viewable via a custom zoombox application. I’m actually trying to reach out to Wikimedia now to get permission to do more bulk downloads as I think my IP got flagged.

Have I topped off what I can get without straight torrenting copyrighted books? I’m even willing to purchase collections if there are methods to do so.

No AI used in the creation of this post. I’ve done a few searches on this subreddit and not come up with anything other than what I’ve mentioned.


r/DataHoarder • • 2h ago

Question/Advice BB easystore - stopped powering on, but if I shuck and use different enclosure, it works

1 Upvotes

As title says. Got a 20TB Best Buy easystore a couple years ago. Didn't shuck it, as it was my dad's drive for use with his shield. Was working fine this whole time.

But now, all of a sudden, it refuses to power on. Plug it in, and nothing happens. Finally shucked the drive out. I tried swapping the little mobo piece and use another one from another shucked drive I had, still doesn't work. Tried multiple different mobo boards from other shucked drives, didn't work.

Finally just plugged it into a generic ugreen external hard drive enclosure I have, and it powers up and works fine.

Making a backup of it now, but not sure what to make of it. SMART details seem fine.


r/DataHoarder • • 1d ago

Question/Advice 12tb of ssd: Server found at Pawn Shop parking lot

138 Upvotes

Hey! I found a super micro server in a pawn shop parking lot.

Would the best way to examine the drives be to load up a Linux distro via a USB drive to look at them and clean them?

They are Samsung 860 QVO 2tb drives.

I don't know the model of the super micro or CPU, but it has 24gb of ddr3 1600 ram.


r/DataHoarder • • 9h ago

Guide/How-to Deleting all Foreign Audio Tracks From Your Collection???

2 Upvotes

I just saved 1.3TB on my storage pool by stripping all the foreign language audio tracks from my movie and TV files. Many of us download Remuxes for the amazing lossless video quality but it also comes with multiple foreign audio tracks depending on where you source them. I know many of you have files that also have Italian audio tracks or Russian that take up space.

Can you imagine how tidious it is to do this manually to every file that fits this criteria let alone pin pointing which even have multile audio tracks? Seems impossible considering thousands of files. I'm here to tell you all this can be done in 1 click. How did I do it?

I used Grok Bot. Grok bot will automate all of this so you just give it the instructions on what you want it to do. It verifies all the tagged files and how much space you'll save. Here are the rules list i've curated with my experience doing this. You could just copy and paste this and see what it tells you for your setup:

FOREIGN AUDIO STRIP RULES

WHAT GETS STRIPPED

- Strip non-English audio tracks from media on (Enter Your Drive Letter) (remuxes included).

- Strip non-English subtitles only from files already being rewritten to remove foreign audio.

Never rewrite a file just to drop foreign subs (saves no real space).

WHAT STAYS

- English audio (main, commentary, descriptive) and English subs, including forced English.

- If a film has no English dub: keep its original-language audio plus English subs.

- Foreign-language films keep their ORIGINAL-language audio even if they also have an English dub

(e.g. Seven Samurai, Ran, Audition, The Raid 2, Grave of the Fireflies, The Lives of Others).

Only extra dubs get stripped, never the film's original language.

HOW IT'S DONE

- Lossless copy only (ffmpeg -c copy): no re-encode, video/HDR/Dolby Vision untouched.

- Each new file is verified (duration, tracks, chapters) before it replaces the original;

if verification fails the original is kept.

AFTERWARD

- Remove language tags (Ita, Rus, 2xRus, Hindi, Korean, Latino, Multi, etc.) from the folder

and main file names so they match the remaining tracks. Keep all other release info.

My scan ran for about 2 days in the background and saved 1.3TB of space. It's worth a look for your drive and see how much you can save for foreign audio tracks you'll never use.


r/DataHoarder • • 7h ago

Guide/How-to Dezoomify-rs vs Wayback Machine websites

1 Upvotes

Hi, I want to download a Zoomify image from a website only available via Wayback Machine, but the browser version couldn't find any zoomable image in the address. Are there other ways how to get such images out?

(I humbly ask you to explain things to me like I'm five)


r/DataHoarder • • 8h ago

Hoarder-Setups U.2 to USB? New life for production retired 2.5" NVME SFF-8639

0 Upvotes

I have some 6.4Tb drives being retired from server duty. I'd like to use them for external drives, large data transfers, local offline backup. To play around with a little.

Samsung 2.5" PM1725B 6 TB NVME PCIE U.2/SFF-8639 form factor

https://www.techpowerup.com/ssd-specs/samsung-pm1725b-6-tb.d2646

12V 2.9A - they need more power than USB, and wrong voltage for or M.2 pcie.

From what I have read U.2 NVME uses the same communication protocol as M.2 NVME and their built in NVME controllers behave the same, o a USB dock/enclosure meant for M.2 might work so long as I provide 12V to the drive.

Could this work?

usb enclosure with M.2 NVME and SATA data/power. I would use the NVME data + sata power only.

https://www.amazon.com/Sabrent-External-Lay-Flat-Docking-EC-DFLT/dp/B0BX4LSHS4/

cable for splitting the connection from U.2 to M.2 data and SATA power

https://www.amazon.com/LetLinkSo-Left-Exit-Motherboard-Workstation/dp/B0DJ8K1VZG/

Or is there a much better solution I'm completely missing? I'm not interested in a large docking station like what Lenovo sells.


r/DataHoarder • • 2d ago

News PSA: Patch your Plex media server before someone hacks it

Thumbnail
arstechnica.com
1.1k Upvotes

Edit: You should patch even if you believe your Plex Media Server is not accessible to the Internet, because a lot of people who thought it wasn't are finding out the hard way that it is!

"...Anyone using Plex should ensure it’s running version 1.43.3. The software can be configured to install updates automatically, or users can manually download and install them. Given the active exploitation, outdated versions should be updated immediately."


r/DataHoarder • • 12h ago

Discussion I'm exploring ideas for logical layer tag-based file system.

1 Upvotes

I'm designing a logical system where the same tag can appear in different contexts without duplicating the tag or the original files. Each assignment preserves its full context, so a work document and a personal photo can both relate to Ferrari without being grouped together in every search.

1. Define a tag once, use it in multiple contexts

Controlled Vocabulary
│
└── Tag Registry                     ← one identity per tag
    ├── Work        [T1]
    ├── Hobbies     [T2]
    ├── Ferrari     [T3]
    ├── Contracts   [T4]
    └── Photography [T5]
              │
              ▼
Tag placements                      ← every label is a tag
│
├── Work [T1]
│   └── Ferrari [T3]
│       └── Contracts [T4]
│
└── Hobbies [T2]
    └── Ferrari [T3]                 ← same tag, different placement
        ├── Photography [T5]         ← child added only here
        └── Contracts [T4]           ← explicitly added here too

The distinction
├── Tag = which concept?
│   └── Ferrari [T3]
│
└── Placement = which context?
    ├── Work → Ferrari
    └── Hobbies → Ferrari

Sharing a tag does not automatically share its children. Adding Photography under Hobbies → Ferrari does not add it under Work → Ferrari. The two Contracts placements also use one shared tag identity, not two separate tags with the same name.

2. Preserve the full context of each assignment

Objects                             ← references to original resources
│
├── Professional agreement [A]
│   └── Work → Ferrari → Contracts
│
├── Model car purchase agreement [B]
│   └── Hobbies → Ferrari → Contracts
│
└── Photo at Monza [C]
    └── Hobbies → Ferrari → Photography

Why keep the full context?
├── Immediate parent only: Ferrari → Contracts
│   └── Cannot distinguish the two agreements
│
└── Full context
    ├── Work → Ferrari → Contracts
    └── Hobbies → Ferrari → Contracts

Assignments refer to stable placement identities, not just strings of names. Renaming a tag does not break them. One object can be assigned to multiple placements without becoming multiple objects.

3. Search everywhere, or stay in one branch

EVERYWHERE: Ferrari
├── [A] Professional agreement
├── [B] Model car purchase agreement
└── [C] Photo at Monza

THIS BRANCH: Work → Ferrari
└── [A] Professional agreement

THIS BRANCH: Hobbies → Ferrari
├── [B] Model car purchase agreement
└── [C] Photo at Monza

EVERYWHERE: Contracts
├── [A] Professional agreement
└── [B] Model car purchase agreement

THIS BRANCH: Hobbies → Ferrari → Contracts
└── [B] Model car purchase agreement

These searches include objects assigned to the selected placements and their descendants. “Everywhere” covers all placements of the selected tag. “This branch” stays within the selected placement and its descendants; it does not jump to other placements of the tags encountered. Each object appears only once in the results.

4. Help users remember the context

Add an object from Work → Ferrari → Contracts
└── Prepare that contextual assignment for confirmation

Add an object from a general entry screen
└── Choose Contracts
    └── Suggest its available contexts
        ├── Work → Ferrari → Contracts
        └── Hobbies → Ferrari → Contracts

Unlike a single folder path, an object can belong to several contexts without being copied. Unlike a flat tag list, the system preserves how those tags were used together. The trade-off is a more complex model, so the interface needs to make the active context obvious.

Would this distinction between a shared tag and its contextual placements be useful in your workflow, or would it add too much friction?


r/DataHoarder • • 23h ago

Question/Advice Add episode number to metadata

3 Upvotes

I have some videos that I want to make sure are ordered in a specific way, aside from just labeling the file Episode 1, Episode 2, etc. I was looking to do it in the metadata, but Windows Properties editor is pretty bare. Is there a way to do this? Is it really necessary? I use Jellyfin, but would like to have the episode number in the file itself in case things change.


r/DataHoarder • • 1d ago

Backup Backup of NAS fails because of 2 different music CDs I ripped

13 Upvotes

I have the Ugreen DXP6800 Pro and use their proprietary Backup & Sync software to backup to a 14TB external HDD. The software allows me to select specific folders to backup, if desired. I got a failure when trying to backup everything... so spent days and hours parsing out individual folders to identify the failure. There are 2 (unrelated, retail purchased) separate music CD's on the entire NAS (currently about 5TB, including 1.5TB of music) that trigger a backup failure. I have now tried ripping them with different software, file formats and editing names and punctuation, and they still trigger a fail. If I leave these 2 album folders unchecked for backup, the rest of my NAS backs up perfectly. I also have checked permissions for these 2 CD folders and nothing obvious. There are hundreds of other albums and artists backed up without issue, plus all else on my NAS. It is very bizarre. Any suggestions what to explore next?

Edit: Resolved! Thanks to uluqat for pushing me to revisit syntax down to the song level, where I finally identified my Backup software does not like the use of "" quotation marks. Plenty of other special characters are used in other song titles without issue... but quotation marks are the problem.


r/DataHoarder • • 1d ago

Question/Advice DAS, modification

Thumbnail
gallery
10 Upvotes

Hi, as the title suggests, i need advice on a DAS modification. The device is FANTEC QB-X8US3-6G.

Its a 8bay bastard, i got it primarilay as i could not get any 4bay nas with modern cpus and tb limits that were higher than 8tb drives or 16tb volumes. All the nas devices that would accept denser storage were out of my budget to put it kindly. So this DAS is gona make do with an old laptop as nas brain. For now, i have populated this with random selection from 10tb to 1 tb drives, some are even 2.5 inch drives. Use case is dead storage, i will offload all the junk of 6 tb from my laptop onto this, whatever is written to this is never gonna get deleted after the initial sift trough. So write once and just read. Than i will transition it into a media server with that aforementioned old laptop.

Now this unit is loud, the hdds arent particularly silent nor loud individualy, but when 7 bastards start to sing in unison at startup its insidious. I need advice on sound dampening materials. What can i do to smother the sound, air comes from front and is exousted in the back. So side panels can be moded somewhat and front door. I would like to change the fans for some quiet ones, fans are 80x80x20, 2 pin j6 connector. I will attach some photos, some of you will want to linch me when they see the orientation, but i simply have no choice in it. It has good ventilation. And no obstruction. Underneath it is a mouse pad to limit the vibration transfer to the desk. I plan to get one for the other side as a buffer between the xbox console and the das.

Any advice is welcome, roasting me like coffe beans is also fine.


r/DataHoarder • • 1d ago

Question/Advice Sorting gallery-dl downloads for Twitter by artist

3 Upvotes

I'm having trouble getting gallery-dl to sort all my likes by artist. Thus far, it just downloads all images into one large folder titled by my twitter handle. I assume there's a way to do this via -o directory= but I'm struggling to figure out what flags I need. Sorry if this problem seems rather straightforward to find, I'm very new to this. I tried reading the documentation, but I honestly couldn't make heads or tails of it.

This is an example of what I have.

gallery-dl -d "F:/ARTWORK" --cookies-from-browser librewolf --date-after '2026-09-01' -o directory={user[id]} "https://x.com/i/history/likes"


r/DataHoarder • • 2d ago

Backup Best Budget-Friendly Way to Store 100–200 TB of 4K Movies, TV Shows, and Documentaries Long-Term?

153 Upvotes

Hey everyone,

I've been building a collection of 4K movies, TV shows, and documentaries for a while now, especially older and hard-to-find content that sometimes disappears from the internet due to copyright issues or streaming services removing it.

Some complete TV series take up 2–3 TB each, and my total collection is around 100–200 TB and growing.

I'm trying to figure out the most affordable and reliable way to preserve everything long-term without spending a fortune.

My priorities:

  • Lowest possible cost, preferably without expensive monthly subscriptions.
  • I don't need constant access. Some files might sit untouched for months.
  • I want to be able to retrieve and watch the original 4K files whenever needed.
  • I want to preserve the original video and audio quality, especially REMUX files, without recompression.
  • Reliability matters. I don't want to lose rare content because of a drive failure or a cloud provider shutting down.
  • I'm open to HDDs, NAS, cloud cold storage, LTO tape, or other alternatives.

I've considered buying large refurbished enterprise HDDs, but storing 100–200 TB gets expensive pretty quickly. Cloud storage also seems costly at this scale.

Someone suggested Blu-ray discs, but managing hundreds or thousands of discs sounds impractical.

For those of you maintaining large 4K media archives, what storage setup do you use? Any recommendations for affordable long-term storage, refurbished enterprise drives, or cloud services that make sense at this scale?

I'm based in the US, so US pricing and retailers would be especially helpful.

Thanks!


r/DataHoarder • • 1d ago

Question/Advice Upgrading drives and migrating data

3 Upvotes

To start off, I tried searching for this and couldn't find much. If I'm just bad at researching, my apologies.

So I have a PC running truenas with 3x12tb drives in raid5. I've come to realize I need more storage, so (hopefully soon) when drive prices become a little more normal, I was going to upgrade to bigger drives. Maybe 3x24tb or similar.

Anyway, I was thinking of how I was actually going to do this. I only have 3 sata ports so installing them all in the same machine won't work. I was thinking I could install the drives into my personal setup and then transfer the data between the two computers.

(Option 1) My main question is could I setup raid5 with the new drives with my current os (fedora Linux if it matters), transfer data, and then put the new drives in my NAS?

(Option 2) I could also install truenas onto my personal rig for this so I'd be transferring between two truenas systems instead if that makes things easier. If I were to do that, would there be an issue with moving the drives into the NAS after data is transferred? I'm not sure if the raid array would stay in tact and if my current NAS would import everything properly.

(Option 3) I've also seen you could replace one drive, rebuild, replace another, rebuild, replace another, rebuild untill all the drives are replaced. This sounds like it introduces more room for failure, but I could be wrong.

(Option 4) My last option that I don't really want to do is to turn my personal PC into my NAS, retire my current NAS, and buy myself a new personal build. This would be the most straightforward to me, but seems kinda wasteful.

I'm not the most experienced with this kind of stuff so I really appreciate the help. Thanks guys!


r/DataHoarder • • 1d ago

Question/Advice Best approach for backing up / snapshotting my PC to my NAS?

1 Upvotes

Basically looking for a way to keep my work backed up so I don’t lose it, in the simplest hands-off way possible, ideally without slowing down my computer.

My PC is Windows 10 LTSC.

My NAS is running TrueNAS

Not sure which tool(s) should I use?

or what's the best way to approach it?


r/DataHoarder • • 1d ago

Question/Advice Search for software

0 Upvotes

I have videos which i move and rename them often. They are on a ugreen nas and i acces them from a macbook. I then have an obsidian from where i link to the videos even though their location or name changes.

I tried to use smb with tailscale and hookmark. Moving files worked but renaming always resulted in broken links. I dont really want to spend so much money on devonthink so maybe someone has an idea.

Linking directly from the nas does not work perfectly because it does not have a good video player.


r/DataHoarder • • 1d ago

Question/Advice any way to extract link from screenshots?

Post image
0 Upvotes

I wanna know if there's any program or page that can extract links from screenshots, ideally give a list of the links contained in the screenshot

for example take the links from this screenshot and list:

tiktok.com/xxxxx
instagram.com/xxxxx

[xxxxx@gmail.com](mailto:xxxxx@gmail.com)
paypal.me/xxxxx

EDIT: I forgot to mention, I need to do this in bulk, like 1000+ screenshots (no joke), so things like AI or image to text are not suitable, since you have to upload the screenshots one by one, 5 max or 10 max


r/DataHoarder • • 1d ago

Backup Deleted backed-up photos/videos from Google Photos on Oct 5. Recovery tool failed. Any ideas?

0 Upvotes

I deleted a bunch of backed-up photos and videos on 5 Oct 2026 by mistake, and they were also deleted from Trash. They were in my Locked Folder with backup turned on. I've restored items from there before using Google's photo recovery tool, but this time it didn't work.

What I've tried:

Checked Trash, Archive and other accounts

Google's photo recovery tool (it failed)

Posted on X to Google Photos (their DM system glitched for me)

My theory: this time I deleted everything in the Locked Folder, leaving it empty, and that might be why the tool found nothing. Tomorrow I'll put one random photo in it and retry.

Questions:

Has anyone had the recovery tool fail, then work on a retry?

Does an empty Locked Folder affect the recovery process?

Is there a Google support route that actually opens a manual check?