MaaStodon't

Posted on 2018-06-03 22:00 in misc

There's this Mastodon thing that's getting talked about. Let's run an instance.
Let's run a few. Let's host many and share the cost.
Let's build a serious fully automated commercial service that can create an instance on demand under 30s. I'll call it MaaStodon.
Maybe i'll finally have something i can call a job and feel safe and stable for a change.

A few technical notes

  • Docker is an okay packaging format. It's bad at everything else. It made my standard Debian kernel panic a few times.
  • Docker Swarm and its overlay network is awful. For instance, once it mixed up a few of its load-balanced IP addresses and sent some requests to the wrong instance. It was fucking terrifying. I didn't sleep well that night; I didn't for a while.
  • Nomad/Consul is nice but immature. Many things i would call critical have been added or changed after I started using it. It's nice to use and watch, less to depend on.
  • Ceph/radosgw is robust and powerful, and also an immense clusterfuck. I wouldn't expect anyone to have any clue about operating it without it being a full time job. I may pay the servers but I'm definitely not in control of the data.

What really went wrong

  • Docker was central to most technical issues. everything related to docker has been an uncontrollable moving part, messing up in ways i couldn't imagine before it happened. It's not a good production tool.
  • Scaling too fast. Masto.host has an order of magnitude more customers managed with a fucking spreadsheet, as i heard. I spent months making powerful software that could scale infinitely and likely never will.
  • Business and what running one implies. I build things, I don't run them.

Conclusion

Managing servers with appropriate structure, resources and boundaries? must be nice.
Being the only developer and sysadmin on call 24/7, and improvised CEO and accountant for a tenth of a decent salary, losing sleep over broken tools? fuck that forever.

I'll keep hosting a few instances on a small-ish server with no moving parts.
What's left of MaaStodon will be given to either social.coop or someone else.
Tools will be archived under some free license. Maybe i'll keep the site private hoping to sell parts later.

Any regret?

Many.