OJ’s rants What would OJ do?

27Dec/0816

The Admin is an Idiot

Yes, and that admin is me!

I usually do a fairly decent job of keeping the server patched, up to date, and clean. Over the Xmas period where I wasn't really looking at anything online, something went wrong.

I logged on this morning only to find that the server wasn't responding. I could connect via SSH, but nothing was being served via HTTP. So I tried to reboot LiteSpeed to see if that would resolve it (as you do ;) ). The service stopped, but failed to restart!

I was a bit concerned when this happened. So I thought I'd check to see if there were any server updates (in case there was a known issue preventing the server from working). I found that there had been an update, so I jumped to my first conclusion and assumed that this was broken. I attempted to download the new version onto my VPS, but it failed. The error message said that it couldn't download the file because I was out of disk space!

A quick run of df told me that the error message was indeed correct. 19GB out of 19GB was used. Crap.

I then jumped to my second conclusion and immediately blamed portage for downloading too much stuff that it didn't need. After a quick eclean I was presented with a huge saving of 350MB. Crap.

I then had to fall back on du (du -h --max-depth=1 to be exact) to try and narrow down the folder(s) which were chewing up disk space.

To cut a long story short, there were two culprits. The first was the linux source folder. I had 15 copies of the Gentoo source in there, and freed up 4.7GB by deleting 13 of them. The second was, much to my shame and dismay, the web server log files folder. Oh woe is me.

Schoolboy error. I scrapped all the unnecessary logs and freed up yet another 5.6GB. Amazingly, everything has started behaving itself again! Who'd have thought.

So the server is now back online. Normality has been restored. I also have more proof that I'm an idiot.

That is all :)

  • OJ
    @Wonky: My comment was as on topic as yours. I just followed your lead.

    Though I have to admit that mine was slightly more loaded ;)
  • Wonky faced ginger monkey
    I have absolutely no idea what on earth you are on about... are you saying I'm fat? Are you saying I am the cake eater. Careful because this is just a taste of the utter drivel I can fill your blog with. I'll bring it down under a tsunami of piffle. You can delete this if you want.
  • OJ
    @Wonkey: I like cake!
  • Wonky faced ginger monkey
    More orange.
  • That's one of the reasons I changed blog software to Movable Type.  Hacking template sets is a lot easier.
  • OJ
    That's what Google is for ;)

    Nah, I did notice that, but I can't be bothered hacking the theme to make it appear. Google does a better job of searching anyway. Cheers for the heads up though!
  • Just noticed that your Wordpress theme doesn't have a search box...  Sort it out lash!
  • TheNapoleon
    No, what I really meant is that he has other things to worry about than thinking he's an idiot, things like, not to put his trousers back to front every morning, putting tea in the sugar rather than the sugar into the tea... anyway, lost in translation me thinks.
  • OJ
    It iz coz you iz winnin' in da crikkit!
  • Iz it coz I iz blak?
  • OJ
    @Rob: That's what I like about you ...
    ... nothing!

    :D
  • @TheNapoleon: Nah-  he's just an idiot ;-)
  • OJ
    @Bryce: Thanks for the info. You're dead right, I didn't download those sources at all as I hadn't planned to update the kernel. Given that this is a VPS I'd be a bit scared of messing something up causing it all to come down in a heap!

    Thanks for the heads up though, I'll be keeping that in mind for future ref. I'm sure others will too :) Cheers!
  • Bryce
    I'm sure you know this, but just in case you don't (or someone else doesn't and just happens to see this post). Just because you download the sources, doesn't mean you use the kernel in gentoo. So don't download the kernel sources unless you plan to upgrade the kernel, just to help save space.

    Obviously that really wasn't your problem here, but just to help save you a step if this happens again.
  • OJ
    @TheNapoleon: Thanks mate. I was overdue for a new coat of paint. Quite like the simplicity of this theme. The other one had JavaScript issues as well, so it's good to get rid of those.

    I don't think I'm totally stupid :) I just like to share my moments of stupidity so that other people can learn from them too!

    Happy New Year :)
  • TheNapoleon
    Love the new web site looks, and don't think too much about being a stupid admin you have other things to worry about.
blog comments powered by Disqus