Mimsy Were the Borogoves

Hacks: Articles about programming in Python, Perl, PHP, and whatever else I happen to feel like hacking at.

Fresh new Negative Space

Jerry Stratton, January 28, 2012

I’ve just moved all of Negative Space over to fresh, new servers on WebFaction, so if you see anything missing, please let me know!

I’ve been running all of my domains except hoboes.com through WebFaction for about a year. The main reasons for the move were the support/price ratio compared to Pair.com, easy support of Django (you will finally see the “temporary” comments system disappearing in favor of the custom system used on The Biblyon Broadsheet due to this), and the ease of adding new domains and new email addresses, as well as the ease of getting this data back out if I need it. WebFaction has an API that lets you script the addition, removal, and listing of most of its services, including email addresses.1

As part of my ongoing move to a Django CMS for all of Negative Space and its satellite domains, I put the last major portions of the site into the CMS last week: my Alexandre Dumas pages and my J. M. Barrie pages. They’re very possibly the earliest content-heavy HTML sections of Negative Space. And they were the last remnant of the old division of Negative Space files between /pub/ and /html/, a practice which dated from the days when ftp sites put their “public” data in /pub/; when the web came around, /pub/ contained mostly text, so I added /html/ as a place to put web pages.

Nowadays, /html/ is the default so there’s no need to clutter up my URL namespaces with “/html” preceding every path. Now that I’ve gotten rid of it, of course, this would be the perfect time for HTML to be superseded by something else…

Because everything’s now in a database, I was able to move the site more easily than in past moves. In the past, I’ve had to do a network copy, which is easy enough, except that I also had to make sure I didn’t move OS-specific files, make sure I didn’t upload originals (which I used Mac OS X’s labels to differentiate), and make sure I didn’t upload files marked as out-of-date or that someone had asked to be removed and which I agreed to remove (again, using labels). Last night, I told the CMS to publish everything that’s marked in the database as hosted on www.hoboes.com, and it did.

The main reason I stuck with Pair despite their higher prices is that my software continued to work; up until last year I used ht://Dig to index the site, because I like being able to do custom search results. Last December, however, an upgrade caused ht://Dig to stop working. Something was blocking ht://Dig from getting web pages after a few minutes of digging, and the Pair engineers had no idea what in the upgrade had changed that would cause that. That’s fine, it’s old software—it won’t even compile on modern systems any more, which of course made it difficult to test—but since ht://Dig and support that knew how their system worked was why I paid Pair’s higher prices, I started looking around for alternatives, and found WebFaction.

It isn’t so much that Pair was bad, it’s just that I was paying a premium price for it—over $30 a month—and service had become pretty standard.2 It seems to me that some hosting providers get stuck: they structure themselves around a pricing model, and when prices start dropping they can’t adapt. The very first hosting provider I used was a company called CTS, in 1995; they don’t seem to exist any more. I paid $300 to set up a dedicated 28.8 connection into my apartment in Pacific Beach, and paid between $175 and $300 a month; it was normally $225, but when my traffic hit 31 megabytes one month they added on a $5 surcharge. Another month, too much spam resulted in a disk overrun of $13.30. As traffic rose, so did the overage charges. At some point I switched from using my own server over a dedicated modem line to using a shell account, and it still ran over a hundred dollars a month. Where, in the beginning, they responded quickly and with knowledgeable people, by the middle they were responding without reading messages, and by the end they weren’t responding at all.

So I moved to Illuminati Online in late 1997, which had a great, and well-deserved, reputation, as well as a $250 charge every three months and, for overages, $45 every month. It wasn’t much of a price drop, but service was vastly greater. But traffic kept going up, and by late 2000 I was using nearly a gigabyte a month, as well as over a hundred megabytes of storage. The monthly charges were now nearly a hundred dollars—not including the $240 per quarter charge. And service was worse; where, at the beginning, I received regular updates and responses from a well-regarded engineer there, the engineer left for a better job, and I ended up having to deal with front-line support that really didn’t know anything about the systems. And so, once again, I started looking around for other solutions, and discovered that I was paying way too much for poor service. In late 2000, I switched to Pair.com.

Pair.com has not suffered nearly as badly as CTS or IO. I have not had to remind them how their system works, or fill my emails to them with caveats about how problems were resolved in the past in order to avoid a lack of institutional memory. But I wasn’t getting the same service as I did when I started; and when I looked around, I found that I could get a much easier-to-use system for much less that does more. I ended up choosing WebFaction.

I’ve been testing it for a year on sites like The Biblyon Broadsheet and The Walkerville Weekly Reader. WebFaction has a much nicer interface for setting up domains and assigning applications to those domains. It was easy, for example, to set up hoboes.com as a separate “short URL” application from www.hoboes.com. On Pair, I had to fake this3 by setting up a folder within www.hoboes.com, watch for requests to hoboes.com in www.hoboes.com and redirect to that folder, and watch for requests to www.hoboes.com in hoboes.com and redirect up.

It’s extremely easy to set up new hostnames: you just type them into a box, and it figures out which domain they belong to. For testing the www.hoboes.com move, I added hobotest.neongoddess.com to neongoddess.com, one of the domains I use for FlameWar: The Passion of the Electric Messiah, and then assigned it to the Negative Space folder. When I was ready to make the move, I also assigned www.hoboes.com to that folder, and when I was sure everything worked, I de-assigned hobotest and removed it from my hostname list.

Another thing WebFaction allows me to easily do is use nginx to serve static content. I have a lot of files inside /library/ and none of it requires PHP or .htaccess files.4 All I needed to do to move it off of Apache and into nginx was set up an nginx application and assign it to www.hoboes.com/library/. The main Negative Space application stops handling /library/ and the new Negative Space Library application starts handling it—faster and more reliably.

They’re missing a couple of things, too.

  • There is no X-Envelope-To on incoming mail, which means client-side rules and Procmail rules have to use a less reliable search of the Received headers.
  • MySQL databases are odd: you get one username/password combo per database, rather than the more standard (and safe) one full-access username, one SELECT-only username, and one SELECT/INSERT/UPDATE/DELETE username. The official workaround is to create a dummy database and use that database’s username as a less-privileged username for real databases.
  • The web servers run under your account’s username and a custom group, which means that bugs in software you deploy (or write yourself) will have read and write access to most everything in your account by default.

All-in-all, though, it’s been a great move—it makes what I’m already doing easier and opens up new possibilities. The Django app for custom comments is one, and the less hacky short URLs for both godsmonsters.com and hoboes.com is another. But there’s a lot more. Their control panel made it easy to set up an A record for my home computer5, and to set up a CNAME to assign a custom host name for a friend’s Blogspot blog. I don’t even understand CNAMEs and A records. And because they’ve got an API for just about everything, including A records, I was able to create a script that runs on my computer whenever my router gets a new IP address. The router triggers the script, which deletes my old A record and adds the new one.

In fact, the main thing I’m worried about with WebFaction is that they make so many things so much easier that, even though it’s easy to get my data out there isn’t going to be anywhere else for me to go.

  1. I copied my list of addresses on Pair from the web browser, cleaned it using Fraise, and then ran a script that created each address in WebFaction via their API.

  2. The worst interaction I had with them was probably when they unilaterally turned off the generic catch-all mailbox I used for hoboes.com. At the time I set up with them—and even when they turned off the catch-all—they didn’t have an easy way to bulk-add email addresses, so I handled it myself in Procmail. I kept in touch with them to make sure this didn’t put too much of a load on their server, and always got the response that it “doesn’t seem to be causing any problems”.

    Until I noticed that mail wasn’t coming in and put in a ticket; it’s possible that they sent me an email to notify me of the change, but since they had effectively removed all of my addresses, it never reached me.

    That was in 2009, however, and at that time ht://Dig still worked, and I knew it would be difficult to recompile on another server.

  3. Pair’s interface let me assign a different folder for hoboes.com and www.hoboes.com—but the hoboes.com setting overrode the www.hoboes.com setting. When I opened up a support ticket for this, the response was that in Apache, “it’s fairly difficult to separate the www and non-www parts of the website. By default, webservers treat them the same”.

    Since I manage Apache in my day job and do exactly this, I knew they were wrong. But I’ve learned that it’s very dangerous to ask your hosting provider to move outside of their comfort zone; they might do it, but because they don’t understand it you’re risking failure down the road. That, also, was another reason to start looking for a better and/or less expensive host.

  4. Except for one April fool joke, but it’s in its own directory so I can make that one directory serve from Apache.

  5. It helps that I discovered Gandi gives me one free SSL certificate per hostname. Since moving to WebFaction I’ve already taken advantage of two of them: one for my home computer as an A record, and one for the Django install. And WebFaction also supports Server Name Indication, which means there is no need to purchase a dedicated IP for each secure site.

  1. <- Tomato
  2. JSKit XML ->