codeaholics.org

addicted to codeahol since 1985

World’s fastest break-in attempt

Apr 16

Posted by Danny in Uncategorized | No Comments | 32 views

Actually, I’m pretty sure it isn’t, but still…

From spinning up a new EC2 instance today to getting the first e-mail from fail2ban took a little under 6 hours.

I don’t know if this makes me happy or sad. Happy because I have a Puppet-based bootstrap system which can bring a freshly minted box up to code in around 5 minutes (including iptables, fail2ban and a locked down SSH configuration), or sad because… well… have people really got nothing better to do?

In related news, when will fail2ban support IPv6? There seem to be lots of threads in lots of different issue tracking systems (most lately Github), many of which include patches, but no actual IPv6 action. Now that makes me sad. :-(

WordPress, nginx, W3TC and robots.txt

Apr 16

Posted by Danny in nginx, Open Source, Performance | No Comments | 48 views

A quick note to try and save somebody else the hours of pain I just experienced…

Here’s the scenario: you’re being dead clever and ditching Apache in favour of Nginx to run your WordPress blog/site and pretty much have everything right. You’re NOT using a plugin to generate robots.txt for you – after all, WordPress does a good enough job through the Settings > Privacy page. You browse to http://domain.com/robots.txt and everything looks pretty sweet. Heck, you might even go and change the privacy settings and grab robots.txt again to make sure it’s all working the way you expect.

Then… you drop the W3 Total Cache bomb. Now, W3TC is pretty well regarded, but it hasn’t had any love for a several months. In fact, it hasn’t even been updated to say it’s compatible with WordPress 3.3.0+ (which it appears to be, AFAICT, although some people have had issues with Minify). What it does have though, is Nginx support out of the box.

What does that mean? Well, if W3TC detects that it is running on Nginx, it will write out a snippet of Nginx configuration which deals with all the cleverness needed to get Nginx to serve W3TC page cache files statically off the disk without having to go through PHP. (This, my friends, is a large part of the secret sauce that makes an Nginx/PHP stack so much faster than Apache/PHP.) Theoretically, all you have to do is use the include directive to pull this snippet into your virtual host configuration file, and you’re good to go. (If you do this then don’t forget to nginx -s reload every time you tweak your W3TC settings.)

And then it hits you. robots.txt has stopped working.

Here’s my solution (in my virtual host file, if you care):

    location = /robots.txt {
        # Force robots.txt through the PHP. This supercedes a match in the
        # generated W3TC rules which forced a static file lookup
        rewrite ^ /index.php;
    }

This is a pretty specific location (using = and not having a regexp), so it trumps anything in the W3TC generated config. Any request for robots.txt is rewritten to index.php which your regular Nginx rules should then hand off to PHP-FPM, which means WordPress will dynamically generate the content for you.

Wow. That took me, literally, 2-3 hours to figure out. Mostly because I didn’t notice it had stopped working when I added W3TC into the mix. Once I’d figured out W3TC (or rather the W3TC generated config) was the culprit, the actual fix was pretty quick.

I’ll be writing more about my Nginx config and the relative performance against Apache2 on an Amazon EC2 Micro instance soon. In the mean time, I hope I saved you some time!

Adventures with DD-WRT and IPv6 (with a dash of TomatoUSB)

Jan 28

Posted by Danny in Uncategorized | No Comments | 314 views

A little under a year ago, I decided two things: first, that it was about time my ageing home network got GigE and 5GHz wireless-N (dual band, of course, to support devices that would only do 2.4GHz); and second, that I would separate the jobs of BEING my network from CONNECTING my network to the internet (since I couldn’t find a good router which would meet these requirements AND had an ADSL modem in it).

So I bought a Linksys/Cisco E3000, made it the backbone of my network and connected it to the internet via my ISP-supplied ADSL modem.

Then an unfortunate incident happened which involved the Linksys/Cisco setup CD, an unwanted but non-removable guest WiFi network, and me swearing a lot.

The time had come (after about 16 hours!) to put DD-WRT on my router. As this post describes, choosing a version of DD-WRT that won’t “brick” your router (as the developers like to describe it) is treacherous to say the least. I eventually settled on dd-wrt.v24-16758_NEWD-2_K2.6_mega (specifically, the nv60k version). Despite the trepidation caused by the dire warnings on the web site, the flashing went well, and I’ve been pleased with DD-WRT ever since. Until…

Last week, I had a 40Mbit/sec fibre broadband connection installed. Amongst other things, my new ISP provides me with a block of IPv6 addresses. Actually 2^80 of them. I seriously need to think about what I’m going to do with them all.

My excitement at having 1,208,925,819,614,629,174,706,176 IP addresses was somewhat dampened when, after a day or so of fiddling and researching, I discovered that DD-WRT’s supposed IPv6 support was limited to the various types of v6-over-v4 tunnels (e.g. Hurricane Electric). Specifically, the PPP daemon doesn’t support IPv6 – so this might just be an issue for PPPoE users. There was no way for me to use all that space natively.

It should be noted here that even if you do want to use a tunnel to reach the IPv6 internet, you will still need to write startup scripts for DD-WRT to load the kernel module (the “Enable IPv6″ checkbox doesn’t actually do anything), start radvd (the “Enable radvd” checkbox doesn’t actually do anything), configure the tunnel interfaces and WAN IP addresses, etc. And even after all of this, you’ll find that the IPv6 user tools (ip6tables, ping6, traceroute6, etc.) aren’t installed, so you’ll have to locate them and hope you have room on your device somewhere.

So the time has come to make the move to TomatoUSB. To some extent, this suffers from the same issues as DD-WRT when it comes to variants, etc., but the information is more logically presented, and there do seem to be fewer choices and fewer potential traps. After looking at the comparison of “mods” on Wikipedia, I chose Toastman’s mod. It seems to have all the features I wanted and he seems to do frequent builds with all the latest updates and patches – in fact, the latest build (1.28.7494.3) was made only 6 days ago. This compares well with DD-WRT which doesn’t appear to have had any real active work/releases for a year or so now.

My first impressions of TomatoUSB are positive. The GUI feels snappy, and has most of the same features as DD-WRT. The real-time bandwidth monitor is definitely prettier than DD-WRTs. And, most importantly, the IPv6 support works out of the box.

TomatoUSB IPv6 configuration screen

Out-of-the-box, ip6tables is configured to allow ICMP packets of every type (so I can ping all my machines from various online ping sites), but disallow all inbound traffic. So, Linux ip6tables bugs aside, I’m secure by default, which is nice. There doesn’t seem to be a GUI interface to setup firewall rules for IPv6, so I guess if I ever to want to let anything in, I’ll have to ssh to the router and do it by hand – but why would I ever want that?

And that’s that. I took under 2 hours to flash TomatoUSB, reproduce all my configuration on it, and get IPv6 working. Nice. I can now browse ipv6.google.com, www.v6.facebook.com/, and I get a dancing turtle when I visit www.kame.net. Also, this:

Results from test-ipv6.com

One last thing: don’t forget to enable IPv6 privacy extensions on all of your hosts!

iTunes 10.5 upgrade woes

Oct 12

Posted by Danny in Uncategorized | 5 Comments | 569 views

iOS 5 is upon us, so I thought I would snag a copy and see what’s what. But, first thing’s first… I apparently needed to upgrade iTunes to 10.5. (Why? Why do I need a particular version of a media player to install a particular version of a mobile phone OS?)

I’m running Windows Vista (yes, really) Ultimate x64 SP2 with all current patches applied. After the obligatory unchecking of unwanted crap from the Apple software update tool (specifically, MobileMe and Safari), I settled in to watch the very slow download. I guess Apple’s servers are a bit overloaded right now. And then the very slow installation process begins. And then… the very slow installation process aborts.

Hmmm. Try again. At least it didn’t seem to need to do the download again. Same failure. No real error message. Just “failed to install” or something equally unhelpful.

A quick Google turned up lots of people having this problem. Some on Windows 7, some on Vista. All on x64. The typical advice was to try installing as an administrator, try downloading and running the MSI by hand, try both (manual install as an administrator). None of it helped. It did, however, reveal a more useful error message. “Service ‘iPod Service’ (iPod Service) could not be installed. Verify that you have sufficient privileges to install system services.”

Googling this turned up a year-old blog post by David Lesault which hit the spot.

Essentially, it seems the installer has issues uninstalling the iPod Service sometimes (I’ve never had this problem before, others seem to have had it since the genesis of iTunes 10.x). The service is marked for deletion, but not quite gone yet. Hence trying to install the new version of the service failed. This is similar to that funky Windows things where it can’t delete files that are in use by a process, but remembers them and deletes them when you reboot. Which, incidentally, is one of the primary reasons why Windows insists on reboots after various kinds of patches, although this is much, much better in Vista and later.

So, I slightly altered David’s process. I got myself to the error message and then simply switched my machine off (hold the power button for 4 seconds). When I restarted and tried the install again, MSI said that a pending installation was in progress and I would need to roll that back before continuing, which I duly did. I was a bit worried that the “rollback” would reinstall the old version of the service, but it didn’t seem to. And that’s that… the iTunes 10.5 upgrade successfully installed.

Now, only 7 minutes left of the iOS 5 download, and who knows how long it will take to actually upgrade the phone and what issues I will have…?

Thanks Apple. :-/

Disruptor.NET

Jul 3

Posted by Danny in Java, Open Source | 3 Comments | 182 views

It’s interesting to see people getting interested in porting the Disruptor to .NET (although what’s wrong with The One True Language, I don’t know!).

I’ll try to keep this post updated as I learn of more .NET interest. Alternatively, please feel free to post a comment below.

The Disruptor – Lock-free publishing

Jun 28

Posted by Danny in Design, Java, Open Source, Performance | 6 Comments | 1,735 views

In case you’ve been living on another planet, we recently open-sourced our high performance message passing framework.

I’m going to give a quick run-down on how we put messages into the ring buffer (the core data structure within the Disruptor) without using any locks.
Read the rest of this entry »

Open sourcing the Disruptor

Jun 21

Posted by Danny in Java, Open Source, Performance | 4 Comments | 177 views

LMAX recently open-sourced The Disruptor – one of the core frameworks upon which we build our ultra-high performance financial exchange. Today, we published a white paper detailing how The Disruptor works, and highlighting the sorts of performance benefits that can be achieved by using it.

The Disruptor is essentially a library which we (and now you!) can use to do message passing within your application. If you like, it’s a queue on steroids. But this stuff is far more fascinating than just that for a number of reasons.
Read the rest of this entry »

The “single implementation” paradox – redux

Jun 19

Posted by Danny in Design, Unit testing | No Comments | 20 views

Following on from my previous post, my colleague, Adrian, has given the issue some thought and written an excellent post explaining his position.

His conclusion is that we should simply not be marking classes as final because it really doesn’t bring any benefits. Even if you’re trying to follow Design by Extension, the next guy is just going to remove the final keyword if it suits his purposes.

Sadly, I suspect he’s right. (Although there’s a conversation to be had here around collective code ownership, and assuming the last person did the best possible job. i.e. that final keyword is maybe there for a reason.)
Read the rest of this entry »

Parallel Ant 0.9 beta released

Jun 16

Posted by Danny in Build, Java, Open Source | No Comments | 4 views

After almost a year with no work, Parallel Ant 0.9 beta is finally released. More info on the Parallel Ant page.

The “single implementation” paradox

Jun 14

Posted by Danny in Design, Unit testing | 2 Comments | 41 views

We got into a bit of a debate at work recently. It went a bit like this:

“Gah! Why do we have this interface when there is only a single implementation?”

(The stock answer to this goes:) “Because we need the interface in order to mock this class in our tests.”

“Oh no you don’t, you can use the FingleWidget [insert appropriate technology of your mocking framework of choice here - e.g. JMock ClassImposteriser]! I’m smarter than you!”

“Well, yes, you can. But if you’ve correctly followed Design for Extension principles, you’ve made the class final, right? And you definitely can’t mock that! Hah! I’m smarter than you!”

“Ah ha! But you could always use the JDave Unfinaliser Agent! I’m so smart it hurts!”
Read the rest of this entry »