Thank you! I actually 3D printed these. I couldn't get my qidi ifast to play nicely with my generic ABS filament, so I sliced them in half and printed them in PLA on my prusa mini. They're joined in the middle with a keyed lock on the top and bottom and 8x 12mm M3 bolts
It's been a fascinating experience so far and I feel like I'm only touching the surface. I'm exploring some of the various memory harnesses to hook into Hermes Agent and see how it learns with extended use.
Currently I'm running a Q6K quant of Hermes 4 14B with a 32K context window via llama.cpp that works pretty well. Generation output is a comfy ~50tok/sec. These v100s are 16GB each, but there are 32GB versions available too.
I'm running everything via NixOS and have to do package overrides to get inference engines to build with the right CUDA versions.
My goal is to get a cohesive environment set up for Hermes Agent to learn my system/lab/network and help my grow it over time.
Overall, I'm happy with them. The mezzanine board is good quality, I'm using PTM sheets under those massive heatsinks and some arctic p9 fans to keep them at around 60C under load.
It's an older version of a heatset insert fixture. I used it to put threaded brass inserts into my 3d prints instead of trying to capture a nut or screwing directly into plastic.
My favorite smell is also my favorite word; Petrichor. It's the smell of the earth when it rains after an extended dry period.
From Wikipedia "from Ancient Greek πέτρα (pétra) 'rock'; or πέτρος (pétros) 'stone' and ἰχώρ (ikhṓr) 'ichor', the ethereal fluid that is the blood of the gods in Greek mythology."
It's a Tesla P40 24gb I got off eBay a while ago. Just like AllHailTheSheep suggested, it's a server GPU meant for compute workflows so it relies on forced air from a server rack. I repasted the die with liquid metal and added a 3d printed duct to attach the radial blower to. The thing never cracks 60c under inference which is nice. Idles around 24c
(Edit: I'm mostly using it for LLMs, but it supposedly makes a good CAD card, so I might give that a shot too)
My janky homelab lol. Mostly ebay secondhand Enterprise stuff and a Chinese SXM2 mezzanine board to run dual NVLinked 16gb V100s. I also have a TrueNAS Scale mini itx server running upstairs with my "arr" stack and some other useful tools.
I felt the same way for a long time. Growing up, I never saw much reason to plan for a future that seemed perilous at best. Climate change was looming; now we are beginning to feel it. The social contract in the US was starting to fray; now it's coming undone.
9/11, the surge of school shootings, multiple "once in a lifetime" recessions, decades of conflict and rampant anti-intellecualism made it seem silly to even consider something like a 401k when it felt like money would be meaningless by the time I retired; if I can retire at all.
Now, I'm content to let the old structures fall. There's a few good or useful pieces in there, but the system as a whole has shown just how hollow or simply rotten some most of them are. I'm not hoping or advocating for collapse, only that if it were to happen in my lifetime, the little corner of humanity I occupy might have the ability to weather the storm.
Someone said it once, "look for the helpers". In times of strife, there's still good to be found.
Thank you! I actually 3D printed these. I couldn't get my qidi ifast to play nicely with my generic ABS filament, so I sliced them in half and printed them in PLA on my prusa mini. They're joined in the middle with a keyed lock on the top and bottom and 8x 12mm M3 bolts