Unplanned Downtime

Don’t ask me why the main server from BuyVM is down. I submitted a ticket, no reply yet.

If I recall correctly, it went down at 6:43 PM.

If I didn’t pay the bill already, I’d consider changing hosts.

Downtime in the next week or so

New Year, New… Intel exploit? Sigh.

Since the launch of our KVM Slices, we have used Intel 1270v3 CPUs with SATA SSD storage. This setup has treated us well with performance that does not break the bank. Unfortunately, Intel CPUs have lost their performance advantage over the last 2 years due to design flaws that have led to security exploits/vulnerabilities.

You would have most likely heard of these exploits/vulnerabilities by the names ‘Meltdown’, ‘Spectre’, and ‘MDS’.

While these exploits/vulnerabilities can be mitigated in software, this comes at the cost of performance. Estimates show a performance loss of around 30% when these mitigations are enabled (and even more if HT/SMT is disabled).

Today we are happy to announce a complete overhaul of all of our KVM Slice nodes to AMD Ryzen 3900X CPUs. Our originally advertised 3.5GHz clocks will increase to 3.8GHz with a boost clock of 4GHz+ (we are seeing an average boost clock of 4.1GHz on most cores). Storage will also be upgraded from SATA SSDs to NVME SSDs, which will increase IO performance by 4x or more.

Network connectivity will be upgraded from 1Gbit to 10Gbit. While your service will remain capped at 1Gbit in each direction, we are considering uncapping inbound speeds for long term customers (customers who have had an active service for >6 months for example). We haven’t given this too much thought yet but many users have shown interest in veteran perks.

Last but not least, A+B power will be set up in Las Vegas to protect us from the Homer Simpsons that always seem to end up working on our power panels. This has been in the works for a while but has only recently become possible.

We have begun ordering equipment for Las Vegas and expect to be racking the new nodes in the next week or so. You will receive an email soon that will document when your VPS will be migrated. A reboot will be required upon which your Slice will automagically be migrated to our new nodes.

We expect to get New York done by early April and will be taking this opportunity to bring our Block Storage Slabs online with new network connectivity (ideally, NYIIX & Hurricane Electric). A new Juniper router will also be installed.

Luxembourg upgrades are for the moment up in the air, but we hope to complete them this year as well.

We thank you for your patronage and are excited to see what you think of our new nodes!

Downtime shouldn’t last long.

DNS Server Unconfigured

The DNS server wasn’t configured since October 27th. Apparently everything cached it for long enough for it to keep working. Debian uses a different folder for the zones.

I found out, because I stopped getting alerts when logging into SSH. Or at least that’s what I think happened, can’t remember. Then I checked my main domain on Leaf DNS, said it wasn’t authoritative. Thought something was wrong with Porkbun, so almost transferred the domain to Namecheap, but cancelled it. Porkbun support told me there was no zone, they were correct.

Just think, if I setup a new server, without copying anything from the backup, the DNS probably would of worked. But, then I’d have to manually restore lots of stuff.

Still Down

How many times do I have to run setup.sh? This might be the last try, then I’ll reinstall Debian, using the ISO, not whatever BuyVM does.

It’s back up, just invalid SSL certificate for some domains.

Down

Got to reboot the server, and restore the backup, not the /home drive, just the root.

I failed at upgrading to CentOS 8, only unofficial methods to do so. What I followed, didn’t work.

Actually, I think, I should say screw it, and switch to Debian. Not allowing me to upgrade the OS without reinstalling, just pisses me off. Can you upgrade Debian to new versions without reinstalling? I think you can.

Update 6:41 PM Going to switch to Debian, soon, after I make a backup of DirectAdmin. If I have to reinstall, I’ll use an OS I prefer, which is Debian.

Oops, I forgot, I probably should of manually installed it. Probably fucked LVM up.

It’ll be a while. Got to resize the partition again. But the /home might be intact.

And, lost all data on the /home drive. Got to copy it all back. Might be done tomorrow.

DirectAdmin is installing, probably should of waited to finish copying the /home data to the /home drive.

And, have to start the setup script all over. I got disconnected from SSH, don’t know how to resume it.

Ran it with screen this time, just like I did with rsync for copying the home files from the backup. I forgot how long it takes to install DirectAdmin.

Don’t think the setup script is working. Might be because the /home is empty, the drive isn’t mounted to /home yet. Also, something needs a newer version of Curl, except it has a newer version, I installed it, probably need the dev package. There we go, that’s installed. Think I’ll wait for it to finish copying the data, before running setup again. Reboot and see if anything works yet. Can’t even login to DirectAdmin.

Update 11:14 PM And DirectAdmin is working, except for PHP. All 404 errors, until it’s finished restoring the files.

Update 12:42 AM Rebooted it, super slow at booting, it’s waiting for a disk. Just need to regenerate the webserver config, and it should work. Still some stuff that needs to be compiled. Since the setup script failed. Oops, SSH doesn’t work, guess I should of checked the home folder for my SSH user before rebooting.

Update 2:24 AM HTTP will be down for a while, I had to run setup.sh again, this time in screen. I’m going to bed, so it may or may not work by itself. There was no FTP, and using CustomBuild didn’t work to get it. Make sure you run DirectAdmin in screen, the installer, otherwise, when you get disconnected, the installer may or may not be doing anything. Got disconnected again. Hopefully screen is still running. It has to compile PHP again, even though it’s already compiled and was working. Sucks, you can’t resume where it crashed or whatever.

Test2

Where’s index.html?

uglyurls = false

Set that to true, and all you get is index.xml.

Haha, it’s /posts.html, not /posts/index.html, probably need to reload with shift.

It Works

Hugo is nice, I should convert my other sites to it. But I’m lazy.

Test

Does it work? Hugo, no PHP or database required.