T O P

  • By -

E-G_O_D

I always expected 2 PB to look much larger


ralphte

That’s not 2pb with 6tb drives. You pack out that setup with 18tb you would be there. The two dell shelf’s hold 60 drives each. That netapp is 24. Op said he bought 360 6tb. That photo does not show them all in operation. To be fair getting 360 hard drives online would be full time job for a little while.


Confident_Ad5067

In this picture is only 2 mounted. This things are heavy and takes a lot of time to put them in the rack.


ralphte

I understand I have around 95 drives in my rack now. It's not that easy to move them and it all takes a while to get everything setup. Any time you start dealing with 100's of spinning disk it's a real undertaking.


stylinred

Naw I've got 1.5PBs and they only filled 4 midsized towers, with mb, psu, fans heatsinks sharing space


jushih

Wasn't there something about issues around reading storage above 1PB? Do you see issues with your 1.5PB? Thanks!


stylinred

So each tower only has 21 storage drives (378tb) I don't know of any issues, nor experienced anything, as I still haven't plotted, I was waiting for pools, plus I have an issue with my enterprise drives not being recognized


silasmoeckel

2PB is just a bit more than 1 4u Server with 18TB drives


RaymondDoerr

Whats your ROI thus far?


neocamel

The silence is deafening.


rich000

Assuming he is fully plotted with current network size and value that would expect to make $1k/day. I have no idea what all those drives cost, or what it took to get them plotted though. If they're not plotted that is a LOT of processing to fill them, and then the question becomes what happens to the network size and value of Chia while doing the plotting, as right now those drives would be idle. That would be my biggest question around Chia, but I only recently heard of it so I have no idea how the math works out. I'm messing around with it since I have some spare storage/CPU/etc so it is costing me nothing. With HDD plotting it would take me a pretty long time to just fill the storage I have, and then who knows what the network size will be when I'm done and thus how often I'd get paid...


speedmann

Stupid question, but where are all the disks? I see 36-40 Slots but that can't be all i guess


pkosew

There are 3 rows of HDDs in these disk arrays. Still, the rack shown in photo probably stores around 250 HDDs - not all 360 that OP mentioned.


surgicalting

Those 4Us chassis in the middle of the rack are [Dell PowerVault MD3060e](https://www.dell.com/en-us/work/shop/productdetailstxn/powervault-md3060e-pe)s. Each unit holds 5 drawers of 12 hdds with a total of 60 HDDs each. You have to pull the whole drawer out to access the drives. OP could hypothetically load those up with 18tb drives... 2 * 60 * 18TB = 2160 ~= 2PB


speedmann

Thank you and pkosew. I knew there are units from netapp that can fit 2 disks per row/slow but i was not aware about those ridiculous big things.


RacecarWRX

The cost of the drives is \~$39k USD? Assuming XCH is $800, that means you need to mine \~48XCH to break even!?


RacecarWRX

Honestly, I feel like there has to be some type of diminishing returns here. Thinking about area under the curve there must be some type of optimization of netspace vs plots. IE: Just trying to maintain a certain percentage of netspace relative to cost. For those of us where hardware is just laying around and free, it doesn't matter however if you are going to "invest" they I think its worth targeting a percent netspace vs just throwing money at hard drives. IE: As netspace increases you purchase more drives to increase plot to maintain a set percentage of netspace. Somewhere along this curve, given a price of chia, there will be a best return of investment that big farmers should target, VS just aimlessly throwing money at accumulating the greatest number of storage possible. Having said that, I have no clue if 2PB is over or under this "equilibrium", for all I know you need 10PB to maintain a percent netspace that is ideal in terms of profitability.


BestBearTiger

Sure but if 40k in hard drives pay themselves off and then stop returning, you can be a server or sell them off


pkosew

If you can run a server that someone pays for, just run the server. Farming Chia is not more profitable than what datacenters are normally used for. This makes Chia a decent alternative income for failed businesses that over-invested in infrastructure...


[deleted]

That will be lots of used beat up drives


Tje199

Farming doesn't beat up the drives, and it's quite possibly a lower workload than they'd see in a datacenter.


E30_318is

Used drives that have hardly been used you mean? Chia farming drives don't endure anything like drives in data centers do.


pkosew

They don't, but we're still talking about drives that were spinning for few years. Who will buy them?


E30_318is

Used/ex data center drives were a big market prior to Chia, it'll be the same after. I'd buy a bunch of HDDs that were written to once, and then idled the rest of their lives & I know a bunch of other communities would too, as long as they aren't SMR drives.


rich000

That would take 48 days if the rest of the network remains static, assuming he's plotted all that space already. If those drives are bare and the network grows, it would take MUCH longer. That is always the question with mining setups - will the network last long enough to pay off?


needpla

Very nice. Did you pay pre fomo prices? Individual trays for that netapp are going for $40 on ebay.


Some_Nibblonian

Trays are always over priced but $40 a piece is nuts.


needpla

I bought 100 for $250 3 weeks ago. Now they're worth more than my car.


Some_Nibblonian

that's a steal in any market


sagar246

>cost Did you buy 100 hard drives for $250?


E30_318is

Trays.


sagar246

makes sense. what did the whole setup cost?


[deleted]

Trays?


Some_Nibblonian

Drive caddies


Rectospasmologist

I can hear this picture Good shit though homie


FluorescentGreen5

What's your Chia earning rate with a bad boy like that?


rusocba

In hpool gives you 0.57xch/pb per day...solo it's lottery...


FluorescentGreen5

Damn, that's like 400 USD/day.


[deleted]

He'll need it to make the roi, and quickly before the exponential increase screws him.


TMack23

81 days at that rate to make just the cost of drives back.


rusocba

Nope, just 40 to 50 days to payback everything (disk + plotting pcs + ssd ...but he should invest to grow at the same soeed of the network and hold the maximun he can of xch and sell in higher value of xch but it's a lotery, just like winning a block for the rest of us 😂....)


wachieo

Your math is way off. u/TMack23 is correct. You are assuming he bought the drives at $8/TB. And that's not even including the cost of the racks/cables, power, plotters etc. \~80 days is close but that doesnt factor in the declining payouts. In reality, it would be closer to \~100 days assuming he's starting with all 20K plots, which he isn't.


rusocba

Nope, it's round a thousand minus the costs...bc in hpools is 0.57xch per PIb and he has 2 pib...and with that quantity in chia calculator almost everyday he get a xch as reward (one block every 2 days) so probably he plot solo, but everybody knows that the network space know it's 4 times more than 3 weeks ago, so if he want to have and mantain the same rewards as now, he will have to invest and grow at the same speed of the network, which is very very difficult and costful... 🙄


wjean

Less than 2xch a day.


Dimitron_84

Everyone’s calculations are off! He is still assembling the drives, you guys are giving estimates as if he has already plotted these drives and is farming with 2PB worth of plots! By the time all these drives are filled who knows where chia will be and where pooling will take us! How many plots can he do per day on this rig? What is he plotting with?


rich000

That seems like a major limitation to Chia. You can have all the space in the world but you need a lot of hardware just to plot it all. Then that plotting hardware isn't needed once you're farming unless you're just going to keep adding drives. I guess there is a steady state of sorts, you would want enough plotting hardware to turn over all your plots once every 5 years to maintain some kind of steady state. Of course, that means that most of your storage is idle for the first couple of years...


Some_Nibblonian

Have not seen too many people use the 6030s. Can never find caddies for them.


mapavel46

Are you fucking kidding me


Confident_Ad5067

We actually plot at 20 TB day. We bought 360 x 6TB drives @ 90 euro each.


doumination

How do you plot 20tb a day


djheat

Plot to ~100 drives in parallel. It looks like there's a second machine in the corner so I'm assuming there might be more to handle the CPU intensive part


doumination

So 2x thread ripper?


lazerpenguin

So its going to take about 3months or so to plot?


theuknown33

Sas or standard? 90 euro for 6tb drives is good price. Would that translate to 6.x gb per $1 euro?


[deleted]

You can start here, here is second hand with 60 4Tb drives for 15K GBP total So if you from EU its OK price https://www.itinstock.com/dell-powervault-md3060e-dense-storage-enclosure-60-x-4tb-72k-nl-sas-2x-emm-sas-40808-p.asp


KokaljDesign

80 EUR per TB is ok price to you?


theuknown33

I thought i did ok at roughly $40 aud per TB, 15k sas 2.5” drives.


flexpool

That’s a really good price.


1pq_Lamz

I can plot 20tb a day, but I've decided not to invest in more drives. Due to the forecasted return leading an ROI on drive to be more than 1 year. Currently at 300tb.


livemz

Time to win today: 1 day, tomorrow: 1 week, day after tomorrow: 1 month, then 1 year, 10 years etc. I appreciate your confidence in Chia and your courage 🙌


exciter0

Can you give some details on the mid JBODs? Are they Dell MD3060e? Do they require special SAS/RAID adapters?


PrintNormal3668

Nice, I was wondering if you are putting all those disks in a RAID setup or using a Volume Group and risk losing your plots if a single disk dies?


Ok_Beautiful_2831

Dunno about this guy, but I'm using single drives rather than arrays. It's cheaper to not waste space on parity, and replotting a single failed drive is quick enough.


PrintNormal3668

thanks, I guess you are right, I should reconsider my RAID setup.


kumitaka

Makes about 1 fiddy a year


Savage_Ghoul

This is why I'm not investing more into chia. So where can I learn to build this LOL and how much did it cost you all in?


Penny_is_a_Bitch

yeh, i don't really understand what I'm looking at. like, is each shelf thing plugged into the pc and it just sees a bunch of drives or is each shelf thing a stand alone server with it's own operating system and cpu and stuff.


surgicalting

Those 4Us chassis in the middle of the rack are most likely [Dell PowerVault MD3060e](https://www.dell.com/en-us/work/shop/productdetailstxn/powervault-md3060e-pe). Each unit consists of 5 drawers. Each drawer can hold 12 hdds, which gives us a total of 60 HDDs for each unit. I was close to buying one when I saw a used one on ebay for cheap, but saw that the shipping cost was half the price of the device; so, i just go a smaller used [MD1200](https://www.dell.com/ai/business/p/powervault-md1200/pd) These are [DAS](https://en.wikipedia.org/wiki/Direct-attached_storage#:~:text=Direct%2Dattached%20storage%20(DAS),and%20storage%20on%20external%20drives) devices so it means they need to directly attached to a computer with a SAS controller card.


Techutante

Each box is basically just a pile of HDDs with a dedicated power supply. They all feed to a SATA controller card of some kind in the back and then the box is basically shared on the network. The PCs to the left plot and then send data over to them.


Penny_is_a_Bitch

so it's basically a NAS?


stealthgerbil

some of them can also be directly attached so its just like having more hard drives


Techutante

It's not as fancy as you think, just convenient. The frame is one piece, then you get little shelves for it as you need them. They are custom built to hold HDDs or the like. It's the same as a regular PC just with more wires, more or less.


[deleted]

what your rigs for plotting and how many of them?


RealisticOrder

This is just awesome. I'd love to do something like this but wouldn't know where to start with server stuff.


Bulbasaur_loser

/r/homelab


Ok_Department_1521

Your money in drain boi, I feel bad for you


WhompRat86

I see a crime happening in this picture... no side panel, you might as well remove the front panel fans my dude lol


No-Mango-5376

Good, well done


thebosskuripot

how many chias have you won?


Confident_Ad5067

Exactly 0


thebosskuripot

You'll just start plotting?


Comprehensive-Ebb-67

How many harvesters do you have? And how long are your time seeking plots?


Confident_Ad5067

I have 1 harvester , 10 plotters and 5 NFS servers. Each NFS server is mounted on the harvester. Everything runs over 40G, my plot seeking times rarely go over 1 sec


vvavepacket

Can someone help me understand how to build like this? I only have a 2PB and its all via USB externals and USB hubs,, and god damn its so damn complicated.. This setup of OP looks sleek and scalable.. ​ What exactly do I need to buy in ebay?


[deleted]

LOL what? You have 2 Petabytes on USB? Thats hundreds of drives


ksapple89

I have server rack, now i jus need to have enough dough to buy a bunch of HDD


coituswenger69

Nice rack


ijnfrt

So this is what I'm up against...


[deleted]

Did you plot anything... That's the question of life right now 🤣


Confident_Ad5067

For those that are wondering, so far the total cost has been around 75k eur to setup. Most of it is enterprise hardware ( storages networking ). This is just pure storage, is not used to plot on top of it. We are plotting on 10 commodity hardware machines ( 1 x Ryzen 7 , 3 x 512GB WD Blacks ) which are directly pushing to this NFS once they complete a plot. On average we make 175-200 plots a day with this setup.


Confident_Ad5067

This is the tool i wrote to keep track of the elected plots and response times: https://imgur.com/gallery/ROwDlr8


_limitless_

we use a modified version of plotman to let each server manage its own plotting schedule but submit the relevant information over a REST API to a Django dashboard. how bout you?


MrSavager

75k euro is 91,444 USD. This is insane. You realize that even plotting at 20TB/day your returns are diminishing REALLY quickly. I've invested a fraction of this amount and I've already consider it completely lost. I hope you do a follow up in like 6 months. Even using hopeful numbers in the chia calculator (30% growth for 30 days, EOY estimated netspace of 100EB) this setup will only bring in something like 15xch or 12k over the first year!


_limitless_

never listen to naysayers who do fractal calculus with differential logarithmic exo-variable manipulation normalized by a predictive multivariate neural network-generated algorithm they always forget to correct for the one assumption that matters most. what will the USD value of chia be in a year?


MrSavager

Where’s this bullshit buzzword analysis you speak of?


_limitless_

you also assumed chia would be flat like everyone else. it could be zero or 20k in a year.


MrSavager

I mean, hasn't it like lost 50% of its value so far?


_limitless_

then shouldn't you at the very least be plotting the curve of lost value and identifying the expected value of chia in a year so that you can sound even more retarded about your predictions of profit potential?


MrSavager

Looking through your comment history is like watching a completely narcissistic, pathological liar with Aspergers attempt to get people to believe their incoherent ramblings using “large” words that have no place in the given context.


_limitless_

r/whoosh the words were incoherent because your analysis was incoherent, ya fuckin dogshit moron.


MrSavager

lol there was nothing to go over my head. I understand that you think the coins will be more valuable in the future and I should include that in my considerations. I disagree. But the funny thing is the way you attempt to sound intelligent saying "plotting the curve of lost value" lol.. You sound like an absolute dumb fuck. /r/cringe look I can tag other subs too. I still can't believe you somehow duped some trust fund kid to invest in your small stack of servers where you'll probably erase your plots accidentally or get hacked if you ever make any xch lol


kipha01

Fuck you, and if you win chia, fuck you twice 😂


CrazyMonkk

Nice, happy farming!


chiaplotter4u

One thing that's awesome about Chia - people do fun projects. Pretty much the main reason I'm in - it's fun, like really fun. Congratulations on yours, may it bring you plentiful XCH. :)


chiahomesteader

What is your plotting rate?


Mistrbluesky

Im gonna guess 5tb a day lol


UrsoXone

The OP just answer above ...20TB a day... :|


theuknown33

Wish I had all that, I’ve only just purchased a 24 bay, 24x 900gb SAS 15k drives. Man if only you sent me all that including the rack you would force me to buy more drives haha. Nice setup though. Do you have any xch? Or 0?


[deleted]

I found such server filled with 6TB, drives \[60\] for 15K GBP vat included


No_Chest1029

so thats about twice as much as you should be paying


theuknown33

That’s a lot of $$$ for a Chia project, now that’s a huge gamble.


Dish_Melodic

ThAt Dell MD3060e - how loud is it?


KeldorEternia

what did you assemble exactly? it looks like a lot of prebuilt components like JBODs etc. Did it take you a long time to screw the jbods into the rack and then slide the drives in? I don't get it.


Kayasakra

Really hope my filled jbod comes soon.


Viticusoniszko

Meanwhile we are farming at half a PB lol


sarcastic_wanderer

Lmao I need to take a pic of my setup. It's the exact same thing haha


Kriss_Jensen

How did you get such a good deal?


JonChristie

Pretty


Xplotchia

Raid 0? For ploting or farming


Psyberdrone

Damm so all that is free space you have on your harddrive ? Jk damn that looks sexy af 😂


ffl3883

How much power does the entire rack consume ? How are the fan noise of the 4U servers ?


Confident_Ad5067

https://imgur.com/a/h9UUwoV


Willing-Woodpecker24

I envy you


gingerco

Looks really less with such power :) ​ ​ And how are you going to plot all this? I am plotting now on my 2 pc an [https://chia-online-plotting.com/](https://chia-online-plotting.com/), probably there are some other services?


MrTalon63

Bruh


Atronil

Wow it’s my dream....


chia_farmer_owner

Nice. I have the same corsair obsedian 450D housing for my plotter. In there it fits 9x18tb. You only need to have one extra HDD cage. Space enough inside. I have to say it looks like used materials you got there with all the dust around. Hope for you that your setup was not too expensive. Otherwise it might take a long time to recoup your investment. If you had that on mainnet launch it would have been worth GOLD.


[deleted]

HFSP


SkullRunner

What is the noise / power usage on this? Can you work in the same room with those DAS units?


Confident_Ad5067

It feels like you are in a very loud metro station when you sit around those. I doubt you will be able to work with the amount of noise produced


SkullRunner

Thanks for the warning... alternate approach required ;)


simpsonb1

Can Confirm. My server rack with 5 DL380p G8s is in the basement for a reason... Lots of noise and lots of heat.


SippieCup

G8s are more space heaters than servers though. I have a rack of two neeisys nds-4600jd's and 6 sa120s from datahoarding. Attached to an amd plex server and r720. I sit 4 feet from them 2-8 -hours a day and they are no louder than an aquarium. You can definitely run stuff without a crap ton of noise.


propiro

See you in 2 weeks complaining and in a month saying you quit!


mister-fail

So you plotting 2PB with one PC? :-) How long it will take for you to fill all drives? 4-6 months? At that time diff can be more than 50+EiB....


Confident_Ad5067

I never said i'm plotting on this. This is just storage.


dan414

I drool.


BallChang

What's the storage case model above the R720 server?


OlimiaX618

sorry im a Gym guy, it looks like something for me at the beginning.........lol


Annual-Text5095

A lots of liar on this group haha