But, why?
Having a dedicated environment to tinker and experiment with random ideas is always nice. Most folks do this on their local environments (laptop/workstations), but it has its limitations:
- Long running jobs can be a pain to setup, especially on a computer that is setup for multiple use cases (eg. dev in the day, game in the evening).
- Experimenting with clusters (multiple nodes) using VMs can get impractical/overwhelming for a single machine.
- Local environments have to be functional at all times, so you cannot do risky changes which might break the environment.
In addition, having separate, always-available compute and storage besides your main dev machine can have the following advantages:
- Ability to self host services, like the following:
- Remote, 1-click-setup coding environments
- Password manager
- Github actions runner for CI/CD
- Photo library
- Local LLM
- Local, fast-access network attached storage (NAS) for backups, large datasets, media storage etc.
- Dev, staging (or even prod, if you are ambitious enough) environments for personal projects.
All of these without the worry of paying $$ to host things in the cloud. According to my estimates, the monthly cost (mainly electricity) of running these machines is 10-20x lower than the same infra in any cloud env.
Here’s a glimpse of a few services I am running as of this moment.
Hardware overview
Compute
3 nodes in a Proxmox (VM hypervisor) cluster. Specs:
| Name | Overview | CPU | RAM | Local SSD storage |
|---|---|---|---|---|
| arete | Dell 5080 Micro | i5-10500T, 6C/12T | 32 GB | 500 GB |
| phoenix | Lenovo M900 Tiny | i5-6500T, 4C/4T | 16 GB | 256 GB |
| thor | Custom PC | R7-3700X, 8C/16T | 64 GB | 256 GB |
So a total of 32vCPUs, 112GB of RAM available.
Storage
The NAS is hosted as a Proxmox VM running TrueNAS. Overall usable capacity of ~10 TB, powered by a pool consisting of 2 mirrored vdevs with the following disks.
| Model | Capacity | Count |
|---|---|---|
| WD DC HC510 | 10TB | 2 |
| Seagate BarraCuda | 1TB | 3 |
The Seagate drives are a leftover from a previous version of this storage array, which I keep running because why not.
Networking
The main device is a TP-Link ER-605, which accepts WAN from two ISPs and handles failover. There’s a 2.5GbE dumb switch which connects to other devices (like my workstation), but everything is still running 1GbE right now. Plan is to upgrade this in the future.