mastodon.ie

Debby ‬⁂📎🐧:disability_flag:Hey everyone 👋I’m diving deeper into running AI models locally—because, let’s be real, the cloud is just someone else’s computer, and I’d rather have full control over my setup. Renting server space is cheap and easy, but it doesn’t give me the hands-on freedom I’m craving.So, I’m thinking about building my own AI server/workstation! I’ve been eyeing some used ThinkStations (like the P620) or even a server rack, depending on cost and value. But I’d love your advice! My Goal: Run larger LLMs locally on a budget-friendly but powerful setup. Since I don’t need gaming features (ray tracing, DLSS, etc.), I’m leaning toward used server GPUs that offer great performance for AI workloads. Questions for the Community: 1. Does anyone have experience with these GPUs? Which one would you recommend for running larger LLMs locally? 2. Are there other budget-friendly server GPUs I might have missed that are great for AI workloads? 3. Any tips for building a cost-effective AI workstation? (Cooling, power supply, compatibility, etc.) 4. What’s your go-to setup for local AI inference? I’d love to hear about your experiences!I’m all about balancing cost and performance, so any insights or recommendations are hugely appreciated. Thanks in advance! 🙌@selfhosted@a.gup.pe <a href="https://hear-me.social/tags/AIServer" class="mention hashtag" rel="nofollow noopener" target="_blank">#AIServer</a> <a href="https://hear-me.social/tags/LocalAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#LocalAI</a> <a href="https://hear-me.social/tags/BudgetBuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#BudgetBuild</a> <a href="https://hear-me.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLM</a> <a href="https://hear-me.social/tags/GPUAdvice" class="mention hashtag" rel="nofollow noopener" target="_blank">#GPUAdvice</a> <a href="https://hear-me.social/tags/Homelab" class="mention hashtag" rel="nofollow noopener" target="_blank">#Homelab</a> <a href="https://hear-me.social/tags/AIHardware" class="mention hashtag" rel="nofollow noopener" target="_blank">#AIHardware</a> <a href="https://hear-me.social/tags/DIYAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#DIYAI</a> <a href="https://hear-me.social/tags/ServerGPU" class="mention hashtag" rel="nofollow noopener" target="_blank">#ServerGPU</a> <a href="https://hear-me.social/tags/ThinkStation" class="mention hashtag" rel="nofollow noopener" target="_blank">#ThinkStation</a> <a href="https://hear-me.social/tags/UsedTech" class="mention hashtag" rel="nofollow noopener" target="_blank">#UsedTech</a> <a href="https://hear-me.social/tags/AICommunity" class="mention hashtag" rel="nofollow noopener" target="_blank">#AICommunity</a> <a href="https://hear-me.social/tags/OpenSourceAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenSourceAI</a> <a href="https://hear-me.social/tags/SelfHostedAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#SelfHostedAI</a> <a href="https://hear-me.social/tags/TechAdvice" class="mention hashtag" rel="nofollow noopener" target="_blank">#TechAdvice</a> <a href="https://hear-me.social/tags/AIWorkstation" class="mention hashtag" rel="nofollow noopener" target="_blank">#AIWorkstation</a> <a href="https://hear-me.social/tags/LocalAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#LocalAI</a> <a href="https://hear-me.social/tags/LLM" class="mention hashtag" rel="nofollow noopener" target="_blank">#LLM</a> <a href="https://hear-me.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener" target="_blank">#MachineLearning</a> <a href="https://hear-me.social/tags/AIResearch" class="mention hashtag" rel="nofollow noopener" target="_blank">#AIResearch</a> <a href="https://hear-me.social/tags/FediverseAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#FediverseAI</a> <a href="https://hear-me.social/tags/LinuxAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#LinuxAI</a> <a href="https://hear-me.social/tags/AIBuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#AIBuild</a> <a href="https://hear-me.social/tags/DeepLearning" class="mention hashtag" rel="nofollow noopener" target="_blank">#DeepLearning</a> <a href="https://hear-me.social/tags/OpenSourceAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#OpenSourceAI</a> <a href="https://hear-me.social/tags/ServerBuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#ServerBuild</a> <a href="https://hear-me.social/tags/ThinkStation" class="mention hashtag" rel="nofollow noopener" target="_blank">#ThinkStation</a> <a href="https://hear-me.social/tags/BudgetAI" class="mention hashtag" rel="nofollow noopener" target="_blank">#BudgetAI</a> <a href="https://hear-me.social/tags/AIEdgeComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#AIEdgeComputing</a> <a href="https://hear-me.social/tags/Questions" class="mention hashtag" rel="nofollow noopener" target="_blank">#Questions</a> <a href="https://hear-me.social/tags/CommunityQuestions" class="mention hashtag" rel="nofollow noopener" target="_blank">#CommunityQuestions</a> <a href="https://hear-me.social/tags/HomeLab" class="mention hashtag" rel="nofollow noopener" target="_blank">#HomeLab</a> <a href="https://hear-me.social/tags/HomeServer" class="mention hashtag" rel="nofollow noopener" target="_blank">#HomeServer</a> <a href="https://hear-me.social/tags/Ailab" class="mention hashtag" rel="nofollow noopener" target="_blank">#Ailab</a> <a href="https://hear-me.social/tags/llmlab" class="mention hashtag" rel="nofollow noopener" target="_blank">#llmlab</a>

Yehor 🇺🇦Almost forgot to add this performance boost to my <a href="https://techhub.social/tags/Jonsbo" class="mention hashtag" rel="nofollow noopener" target="_blank">#Jonsbo</a> N1 DIY NAS.I think I can conclude that <a href="https://techhub.social/tags/TheGRATEmigration11" class="mention hashtag" rel="nofollow noopener" target="_blank">#TheGRATEmigration11</a> was successful. <a href="https://techhub.social/tags/TrueNAS" class="mention hashtag" rel="nofollow noopener" target="_blank">#TrueNAS</a> with *arr stack is up and running. My iso images are re-downloading, <a href="https://techhub.social/tags/Plex" class="mention hashtag" rel="nofollow noopener" target="_blank">#Plex</a> transcoding performance is amazing.I need to migrate some services from a mini PC to TrueNAS. I also need a reverse proxy for them. I also meed to set up a backup solution for TrueNAS… There is still a lot to do. As usual in the <a href="https://techhub.social/tags/homelab" class="mention hashtag" rel="nofollow noopener" target="_blank">#homelab</a>But now I rest. I <a href="https://techhub.social/tags/vacation" class="mention hashtag" rel="nofollow noopener" target="_blank">#vacation</a> and <a href="https://techhub.social/tags/Diablo" class="mention hashtag" rel="nofollow noopener" target="_blank">#Diablo</a> IV. I also <a href="https://techhub.social/tags/beer" class="mention hashtag" rel="nofollow noopener" target="_blank">#beer</a> and <a href="https://techhub.social/tags/snacks" class="mention hashtag" rel="nofollow noopener" target="_blank">#snacks</a>. Stay tuned. <a href="https://techhub.social/tags/pcbuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#pcbuild</a> <a href="https://techhub.social/tags/serverbuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#serverbuild</a> <a href="https://techhub.social/tags/jonsbon1" class="mention hashtag" rel="nofollow noopener" target="_blank">#jonsbon1</a> <a href="https://techhub.social/tags/selfhosted" class="mention hashtag" rel="nofollow noopener" target="_blank">#selfhosted</a> <a href="https://techhub.social/tags/selfhosting" class="mention hashtag" rel="nofollow noopener" target="_blank">#selfhosting</a> <a href="https://techhub.social/tags/selfhost" class="mention hashtag" rel="nofollow noopener" target="_blank">#selfhost</a> <a href="https://techhub.social/tags/homeserver" class="mention hashtag" rel="nofollow noopener" target="_blank">#homeserver</a> <a href="https://techhub.social/tags/pcbuilding" class="mention hashtag" rel="nofollow noopener" target="_blank">#pcbuilding</a> <a href="https://techhub.social/tags/diynas" class="mention hashtag" rel="nofollow noopener" target="_blank">#diynas</a> <a href="https://techhub.social/tags/nas" class="mention hashtag" rel="nofollow noopener" target="_blank">#nas</a>

Yehor 🇺🇦Just some <a href="https://techhub.social/tags/homelab" class="mention hashtag" rel="nofollow noopener" target="_blank">#homelab</a> beauty during <a href="https://techhub.social/tags/TheGRATEmigration11" class="mention hashtag" rel="nofollow noopener" target="_blank">#TheGRATEmigration11</a> <a href="https://techhub.social/tags/jetkvm" class="mention hashtag" rel="nofollow noopener" target="_blank">#jetkvm</a> <a href="https://techhub.social/tags/serverbuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#serverbuild</a>

Yehor 🇺🇦And all drives are detected! That's a win! <a href="https://techhub.social/tags/homelab" class="mention hashtag" rel="nofollow noopener" target="_blank">#homelab</a> <a href="https://techhub.social/tags/TheGRATEmigration11" class="mention hashtag" rel="nofollow noopener" target="_blank">#TheGRATEmigration11</a> <a href="https://techhub.social/tags/pcbuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#pcbuild</a> <a href="https://techhub.social/tags/pcbuilding" class="mention hashtag" rel="nofollow noopener" target="_blank">#pcbuilding</a> <a href="https://techhub.social/tags/serverbuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#serverbuild</a>

Yehor 🇺🇦We have green light! It’s alive! Alive!!! <a href="https://techhub.social/tags/pcbuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#pcbuild</a> <a href="https://techhub.social/tags/pcbuilding" class="mention hashtag" rel="nofollow noopener" target="_blank">#pcbuilding</a> <a href="https://techhub.social/tags/TheGRATEmigration11" class="mention hashtag" rel="nofollow noopener" target="_blank">#TheGRATEmigration11</a> <a href="https://techhub.social/tags/homelab" class="mention hashtag" rel="nofollow noopener" target="_blank">#homelab</a> <a href="https://techhub.social/tags/serverbuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#serverbuild</a> <a href="https://techhub.social/tags/homeserver" class="mention hashtag" rel="nofollow noopener" target="_blank">#homeserver</a> <a href="https://techhub.social/tags/jonsbo" class="mention hashtag" rel="nofollow noopener" target="_blank">#jonsbo</a> <a href="https://techhub.social/tags/jonsbon1" class="mention hashtag" rel="nofollow noopener" target="_blank">#jonsbon1</a>

Yehor 🇺🇦The PSU for my <a href="https://techhub.social/tags/DIYNAS" class="mention hashtag" rel="nofollow noopener" target="_blank">#DIYNAS</a> finally arrived and I’ve put everything together. Ready for the first launch. My biggest concern is HBA SAS controller. Will it work? I have total of seven storage drives connected. Hope to see them all in BIOS setup. Hope I’ll be able to get to the BIOS setup )) <a href="https://techhub.social/tags/TheGRATEmigration11" class="mention hashtag" rel="nofollow noopener" target="_blank">#TheGRATEmigration11</a> <a href="https://techhub.social/tags/pcbuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#pcbuild</a> <a href="https://techhub.social/tags/pcbuilding" class="mention hashtag" rel="nofollow noopener" target="_blank">#pcbuilding</a> <a href="https://techhub.social/tags/diy" class="mention hashtag" rel="nofollow noopener" target="_blank">#diy</a> <a href="https://techhub.social/tags/serverbuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#serverbuild</a> <a href="https://techhub.social/tags/nas" class="mention hashtag" rel="nofollow noopener" target="_blank">#nas</a> <a href="https://techhub.social/tags/homeserver" class="mention hashtag" rel="nofollow noopener" target="_blank">#homeserver</a> <a href="https://techhub.social/tags/jonsbo" class="mention hashtag" rel="nofollow noopener" target="_blank">#jonsbo</a> <a href="https://techhub.social/tags/jonsbon1" class="mention hashtag" rel="nofollow noopener" target="_blank">#jonsbon1</a> <a href="https://techhub.social/tags/homelab" class="mention hashtag" rel="nofollow noopener" target="_blank">#homelab</a>

Debby ‬⁂📎🐧:disability_flag:<a href="https://palaver.p3x.de/u/hendrik" class="u-url mention" rel="nofollow noopener" target="_blank">@hendrik</a> <a href="https://feddit.org/u/pandasiusfilet" class="u-url mention" rel="nofollow noopener" target="_blank">@pandasiusfilet</a> Hey there! 😊 Sorry to bother you, but I’m considering upgrading my homelab with some P40 or M100 GPUs. I mainly need more VRAM for my projects, and I was really inspired by the channel "Homelab AI" and their video on building a DIY 4x Nvidia P40 homeserver with 96GB of VRAM! If you haven’t seen it yet, you can check it out here: DIY 4x Nvidia P40 Homeserver for A <a href="https://youtu.be/dHTvpUlWFbk" rel="nofollow noopener" translate="no" target="_blank">https://youtu.be/dHTvpUlWFbk</a> . I’d love to hear your thoughts or any tips you might have? Thanks a lot! 🙌 <a href="https://hear-me.social/tags/ai" class="mention hashtag" rel="nofollow noopener" target="_blank">#ai</a> <a href="https://hear-me.social/tags/machinelearning" class="mention hashtag" rel="nofollow noopener" target="_blank">#machinelearning</a> <a href="https://hear-me.social/tags/artificialintelligence" class="mention hashtag" rel="nofollow noopener" target="_blank">#artificialintelligence</a> <a href="https://hear-me.social/tags/Homelab" class="mention hashtag" rel="nofollow noopener" target="_blank">#Homelab</a> <a href="https://hear-me.social/tags/AI" class="mention hashtag" rel="nofollow noopener" target="_blank">#AI</a> <a href="https://hear-me.social/tags/Nvidia" class="mention hashtag" rel="nofollow noopener" target="_blank">#Nvidia</a> <a href="https://hear-me.social/tags/GPUs" class="mention hashtag" rel="nofollow noopener" target="_blank">#GPUs</a> <a href="https://hear-me.social/tags/P40" class="mention hashtag" rel="nofollow noopener" target="_blank">#P40</a> <a href="https://hear-me.social/tags/M100" class="mention hashtag" rel="nofollow noopener" target="_blank">#M100</a> <a href="https://hear-me.social/tags/VRAM" class="mention hashtag" rel="nofollow noopener" target="_blank">#VRAM</a> <a href="https://hear-me.social/tags/DIY" class="mention hashtag" rel="nofollow noopener" target="_blank">#DIY</a> <a href="https://hear-me.social/tags/TechProjects" class="mention hashtag" rel="nofollow noopener" target="_blank">#TechProjects</a> <a href="https://hear-me.social/tags/MachineLearning" class="mention hashtag" rel="nofollow noopener" target="_blank">#MachineLearning</a> <a href="https://hear-me.social/tags/DataScience" class="mention hashtag" rel="nofollow noopener" target="_blank">#DataScience</a> <a href="https://hear-me.social/tags/ServerBuild" class="mention hashtag" rel="nofollow noopener" target="_blank">#ServerBuild</a> <a href="https://hear-me.social/tags/TechInspiration" class="mention hashtag" rel="nofollow noopener" target="_blank">#TechInspiration</a> <a href="https://hear-me.social/tags/HomeServer" class="mention hashtag" rel="nofollow noopener" target="_blank">#HomeServer</a> <a href="https://hear-me.social/tags/CloudComputing" class="mention hashtag" rel="nofollow noopener" target="_blank">#CloudComputing</a>

Recent searches

Search options

Administered by:

Server stats:

#serverbuild