markhneedham/blog huggingface dataset:
Ruby/Haml: Maintaining white space/indentation in a <pre> tag
Date: 2013-03-02 22:19:11

I've been writing a little web app in which I wanted to display cypher queries inside a

....
 tag which was then prettified using SyntaxHighlighter but I was having problems with how code on new lines was being displayed.</p>


I had the following Haml code to display a query looking up Gareth Bale in a graph:



~~~haml

%pre{ :class => "brush: cypher; gutter: false; toolbar: false;"}
  START player = node:players('name:"Gareth Bale"')
  RETURN player.name
~~~

When I rendered the page it looked like this:






After a bit of googling I ended up on this Stack Overflow post which described the preserve helper which seems to do the job:



~~~haml

%pre{ :class => "brush: cypher; gutter: false; toolbar: false;"}
  = preserve do
    START player = node:players('name:"Gareth Bale"')
    RETURN player.name
~~~

That part of the page now looks much better:
....
Ruby/Haml: Conditionally/Optionally setting an attribute/class
Date: 2013-03-02 23:22:50

One of the things that we want to do reasonably frequently is set an attribute (most often a class) on a HTML element depending on the value of a variable.

I always forget how to do this in Haml so I thought I better write it down so I'll remember next time!

Let's say we want to add a +++<cite>+++success+++</cite>+++ class to a paragraph if the variable +++<cite>+++correct+++</cite>+++ is true and not have any value if it's false.

The following code does what we want:

[source,haml]
----

- correct = true
%p{:class => (correct ? "success" : nil) }
  important text
----

This generates the following HTML is +++<cite>+++correct+++</cite>+++ is true:

[source,html4strict]
----

<p class="success">
  important text
</p>
----

And the following HTML if it's false

[source,html4strict]
----

<p>
  important text
</p>
----

To summarise, http://stackoverflow.com/questions/3841116/conditionally-set-html-element-id-with-haml[if we set an attribute to nil in Haml it just won't be rendered at all] which is exactly what we want in this situation.
Editing config files on a server & Ctrl-Z
Date: 2013-03-29 10:51:37

A couple of weeks ago https://twitter.com/timrgoodwin[Tim] and I were spinning up a new service on a machine which wasn't quite working so we were manually making changes to the +++<cite>+++/etc/nginx/nginx.conf+++</cite>+++ file and restarting nginx to try and sort it out.

This process is generally not that interesting - you open the file in vi, make some changes, close it, then restart nginx and see if it works. If not then you open the file again and repeat.

Except Tim had a slight variation on this workflow which is an improvement that I don't want to forget!

Once we'd finished making the changes to the file in vi Tim hit 'http://en.wikipedia.org/wiki/Control-Z[Ctrl + Z]' which suspended the vi process and put us back at the shell prompt.

We could then restart nginx or do whatever else we needed to do and then type 'http://en.wikipedia.org/wiki/Fg_(Unix)[fg]' to go back into vi again.

Not only is this workflow quicker, it also keeps the history of the changes that we've made to the file so if one of our changes really screws things up we can easily undo it. Previously we'd have to remember what changes we'd made and do that manually.

In summary this workflow is a pretty simple idea but nevertheless one I had never thought about or seen anyone else do and I'll be using it in future.
A quick and dirty way of testing the performance of a service
Date: 2013-03-16 11:58:42

We had a power outage in our data centre yesterday and once it had recovered https://twitter.com/jasonneylon[Jason] and I wanted to do a quick check that one of our backend services was still responding in an acceptable amount of time.

Since this particular service only serves HTTP GET requests it was reasonably easy to http://unixbhaskar.wordpress.com/2010/11/12/measure-website-response-time-through-curl/[setup a cURL command to do this]:

[source,text]
----

while true; do curl -k -s -w %{time_total}  https://serviceurl/whatever/something; -o /dev/null; printf "\n"; done > service.txt
----

The '-w' flag is one I haven't used before but it allows us to output a bunch of interesting things about the request.

In this case we're only interested in the end to end time of the request/response but we could be more fine grained if we wanted to be.

We ran that for a little while and then killed it and wrote a little Ruby script to get the times in a format that would make it easy to plug into R:

[source,ruby]
----

# output cut for brevity
$ p File.readlines("service.txt").map { |x| x.gsub(/\n/, "").to_f }.sort
[0.042, 0.043, 0.043, 0.045, 0.046...1.083]
----

We then plotted a Histogram in R:

[source,r]
----

> times = c(0.042, 0.043, 0.043, 0.045, 0.046, 0.046, 0.046, 0.047, 0.047, 0.047, 0.048, 0.048, 0.048, 0.048, 0.048, 0.048, 0.048, 0.049, 0.05, 0.05, 0.05, 0.051, 0.051, 0.051, 0.051, 0.051, 0.051, 0.051, 0.051, 0.052, 0.052, 0.052, 0.053, 0.053, 0.053, 0.054, 0.054, 0.055, 0.055, 0.056, 0.056, 0.056, 0.056, 0.056, 0.056, 0.056, 0.056, 0.057, 0.057, 0.057, 0.057, 0.058, 0.059, 0.06, 0.06, 0.06, 0.061, 0.061, 0.061, 0.061, 0.061, 0.061, 0.062, 0.062, 0.063, 0.063, 0.064, 0.066, 0.066, 0.067, 0.071, 0.071, 0.072, 0.073, 0.074, 0.079, 0.081, 0.084, 0.085, 0.085, 0.091, 0.108, 0.116, 0.14, 0.182, 0.26, 0.281, 1.083)
> hist(times)
----

image::{{<siteurl>}}/uploads/2013/03/service-tiff.jpg[Service tiff,600]

That roughly shows us that the massive majority of requests were being served in under 1/10th a second which was acceptable for this service.

We can also run a couple of other commands on the times to learn more about the distribution:

[source,r]
----

> summary(times)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
0.04200 0.05100 0.05600 0.07745 0.06325 1.08300
----

[source,r]
----

> quantile(times, 0.95)
   95%
0.1316
----

From these functions we learn that the max value is an outlier compared to the others and 95% of the requests were served in 0.13 seconds or less which was enough evidence to convince us that the service hadn't degraded.
clojure/Java Interop: The doto macro
Date: 2013-03-17 20:21:10

I recently wrote about some http://www.markhneedham.com/blog/2013/03/17/clojurejava-interop-importing-neo4j-spatial-data/[code I've been playing with to import neo4j spatial data] and while looking to simplify the code I came across the +++<cite>+++http://clojure.org/java_interop#Java Interop-The Dot special form-(doto instance-expr (instanceMethodName-symbol args*)*)[doto]+++</cite>+++ macro.

The +++<cite>+++doto+++</cite>+++ macro allows us to chain method calls on an initial object and then returns the resulting object. e.g.

[source,lisp]
----

(doto (new java.util.HashMap) (.put "a" 1) (.put "b" 2))
-> {a=1, b=2}
----

In our case this comes in quite useful in the function used to create a stadium node which initially reads like this:~~~ ~~~lisp (defn create-stadium-node [db line] (let [stadium-node (.. db createNode)] (.. stadium-node (setProperty "wkt" (format "POINT(%s %s)" (:long line) (:lat line)))) (.. stadium-node (setProperty "name" (:stadium line))) stadium-node)) ~~~

Here we first create a node, set a couple of properties on the node and then return it.

Using the macro it would read like this: ~~~lisp (defn create-stadium-node [db line] (doto (.. db createNode) (.setProperty "wkt" (format "POINT(%s %s)" (:long line) (:lat line))) (.setProperty "name" (:stadium line)))) ~~~

We can also use it to close the transaction at the end of our function although we don't actually have a need for the transaction object which gets returned:~~~ ~~~lisp # the end of our main function (.. tx success) (.. tx finish) ~~~

\...becomes... ~~~lisp (doto tx (.success) (.finish)) ~~~

As far as I can tell this is pretty similar in functionality to the +++<cite>+++http://blog.moertel.com/posts/2007-02-07-ruby-1-9-gets-handy-new-method-object-tap.html[Object#tap]+++</cite>+++ function in Ruby: ~~~ruby {}.tap { |x| x[:a] = 1; x[:b] = 2 } \=> {:a\=>1, :b\=>2} ~~~

Either way it's a pretty neat way of simplifying code.
Understanding what lsof socket/port aliases refer to
Date: 2013-03-17 14:00:35

Earlier in the week we wanted to check which ports were being listened on and by what processes which we can do with the following command on Mac OS X:

[source,text]
----

$ lsof -ni | grep LISTEN
idea       2398 markhneedham   58u  IPv6 0xac8f13f77b903331      0t0  TCP *:49410 (LISTEN)
idea       2398 markhneedham   65u  IPv6 0xac8f13f7799a4af1      0t0  TCP *:58741 (LISTEN)
idea       2398 markhneedham  122u  IPv6 0xac8f13f7799a4711      0t0  TCP 127.0.0.1:6942 (LISTEN)
idea       2398 markhneedham  249u  IPv6 0xac8f13f777586711      0t0  TCP *:63342 (LISTEN)
idea       2398 markhneedham  253u  IPv6 0xac8f13f777586331      0t0  TCP 127.0.0.1:63342 (LISTEN)
java      16973 markhneedham  152u  IPv6 0xac8f13f777586af1      0t0  TCP *:56471 (LISTEN)
java      16973 markhneedham  154u  IPv6 0xac8f13f779e6b711      0t0  TCP *:menandmice-dns (LISTEN)
java      16973 markhneedham  168u  IPv6 0xac8f13f77b902f51      0t0  TCP 127.0.0.1:7474 (LISTEN)
java      16973 markhneedham  171u  IPv6 0xac8f13f77b013711      0t0  TCP 127.0.0.1:7473 (LISTEN)
----

One of the interesting things about this output is that for the most part it shows the port number and which IPs it will accept a connection from but sometimes it uses a socket/port alias.

In this case we can see that the 3rd last line refers to 'menandmice-dns' but others could be 'http-alt' or 'mysql'.

We can find out what port those names refer to by looking in +++<cite>+++/etc/services+++</cite>+++:

[source,text]
----

$ cat /etc/services | grep menandmice-dns
menandmice-dns  1337/udp    # menandmice DNS
menandmice-dns  1337/tcp    # menandmice DNS
----

[source,text]
----

$ cat /etc/services | grep http-alt
http-alt	591/udp     # FileMaker, Inc. - HTTP Alternate (see Port 80)
http-alt	591/tcp     # FileMaker, Inc. - HTTP Alternate (see Port 80)
http-alt	8008/udp     # HTTP Alternate
http-alt	8008/tcp     # HTTP Alternate
http-alt	8080/udp     # HTTP Alternate (see port 80)
http-alt	8080/tcp     # HTTP Alternate (see port 80)
----

There's a http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xml[massive XML document on the IANA website] with a full list of the port assignments which is presumably where +++<cite>+++/etc/services+++</cite>+++ is derived from.
Wiring up an Amazon S3 bucket to a CNAME entry - The specified bucket does not exist
Date: 2013-03-21 22:39:02

https://twitter.com/jasonneylon[Jason] and I were setting up an internal static website using an http://aws.amazon.com/s3/[S3] bucket a couple of days ago and wanted to point a more friendly domain name at it.

We initially called our bucket 'static-site' and then created a http://en.wikipedia.org/wiki/CNAME_record[CNAME] entry using http://www.zerigo.com/[zerigo] to point our sub domain at the bucket.

The mapping was something like this:

[source,text]
----

our-subdomain.somedomain.com -> static-site.s3-website-eu-west-1.amazonaws.com
----

When we tried to access the site through our-subdomain.somedomain.com we got the following error:

[source,text]
----

<Error>
<Code>NoSuchBucket</Code>
<Message>The specified bucket does not exist</Message>
<BucketName></BucketName>
<RequestId></RequestId>
<HostId>
----

A bit of googling led us to https://forums.aws.amazon.com/message.jspa?messageID=237562[this thread] which suggested that we needed to ensure that our bucket was named after the sub domain that we wanted to serve the site from.

In this case we needed to rename our bucket to 'our-subdomain.somedomain.com" and then our CNAME entry to:

[source,text]
----

our-subdomain.somedomain.com -> our-subdomain.somedomain.com.s3-website-eu-west-1.amazonaws.com
----

And then everything was happy.
Embracing the logs
Date: 2013-03-31 21:44:19

Despite the fact that I've been working full time in software for almost 8 years now every now and then I still need a reminder of how useful reading logs can be in helping solve problems.

I had a couple of such instances recently which I thought I'd document.

The first was a couple of weeks ago when https://twitter.com/timrgoodwin[Tim] and I were pairing on http://www.markhneedham.com/blog/2013/03/24/incrementally-rolling-out-machines-with-a-new-puppet-role/[moving some applications from Passenger to Unicorn] and were testing whether or not we'd done so successfully.

We were doing this by creating an +++<cite>+++/etc/hosts+++</cite>+++ entry from our top level domain to an nginx proxy node which was to forward on the request to the application server.

[source,text]
----

Request -> nginx on proxy node -> nginx on app server node -> unicorn on app server node
----

This didn't work and we got a 404 response code so I logged onto the server hosting the application server and started writing our a cURL command to simulate what the proxy should be doing to see if the problem was there.

After watching me do this a couple of times Tim suggested that we might be more successful if we opened a bunch of tabs on the shell tailing the various log files that the request should pass through.

We set up tail commands against the following files:

* nginx access log on proxy node
* nginx error log on proxy node
* nginx access log on the app server node
* unicorn log on the app server node

Immediately it became clear that we actually had a problem on the proxy node because we'd configured one of the nginx directives incorrectly.

Once we'd fixed this the request flowed through smoothly.

We extended this tailing of files idea when testing multiple nodes through a load balancer except this time we http://www.markhneedham.com/blog/2013/01/15/fabric-tailing-log-files-on-multiple-machines/[made use of Fabric] to make things a bit easier.

The second was earlier this week when https://twitter.com/barisbalic[Baris] and I were trying to configure puppet so that we could install different Ruby versions on different machines.

We were having trouble figuring out why the wrong version was getting installed so eventually we chatted to https://twitter.com/supersheep[Andy] who amongst other things had a look at the apt history log @ +++<cite>+++/var/log/apt/history.log+++</cite>+++ and was able to figure out how this was happening.

Lesson of the last two weeks: *embrace the logs*!
Best tool for the job/Learning new ways to do things
Date: 2013-03-24 22:01:12

I recently came across an interesting post written by https://twitter.com/Me1000[Randy Luecke] titled 'http://randyluecke.tumblr.com/post/45915323813/im-done-with-the-web[I'm done with the web]' in which he expresses his surprise that people often aren't willing to take the time out to learn something new.

In this context he's referring to javascript libraries but I think his thinking is generally applicable.

Having worked for a few years now I've played around with a reasonable number of programming languages/text editors/databases/etc to the point that I have favourites when it comes to solving certain problems.

If I come across a new problem and I want to solve it quickly my first instinct is to use one of the things that I know and am comfortable with as this will allow me to solve the problem most quickly.

The problem with this way of thinking is that it's restrictive - *what if there's some other way of solving the problem which is actually better than what I'm doing* but would take some practice for me to recognise that?

For example at the moment if I wanted to hack together a script which used a Java API I'd probably just crack open IntelliJ and write the code in Java even if http://www.markhneedham.com/blog/2013/03/17/clojurejava-interop-importing-neo4j-spatial-data/[clojure might be a better choice] if I knew it better.

If I want to quickly put together a simple web application I'd use http://www.sinatrarb.com/[Sinatra] but would http://flask.pocoo.org/[one] https://github.com/breily/juno[of the] http://bottlepy.org/docs/dev/index.html[Python equivalents] be better if I was more familiar with Python or does it not matter?

I'm reminded of conversations that I used to have with http://www.oshineye.com/[Ade] when we worked together and he was showing me http://www.amazon.co.uk/Apprenticeship-Patterns-Guidance-Aspiring-Craftsman/dp/0596518382/ref=sr_1_1?ie=UTF8&qid=1364161683&sr=8-1[Apprenticeship Patterns] and in particular the 'http://ofps.oreilly.com/titles/9780596518387/emptying_the_cup.html#confront_your_ignorance[Confront your ignorance]' pattern.

____
You have identified gaps in your skillset, gaps that are relevant to your daily work.
____

In this case the gap I'm talking about is relevant in as far as you can arguably do a better job if you have more tools to choose from.

A common pattern that I've noticed myself/others doing is to stick with the toolset we're comfortable with until it breaks down at which point we start looking around for something else.

While this approach does work it feels like we should be more proactive in learning new things so that we can question whether are current tools are actually the best ones for the job rather than just assuming they are.

I don't really have a conclusion other than that Randy's post struck a chord and I want to try and always challenge myself to keep learning things.

It'd be interesting to know if others have a similar experience or if I'm way off track?
Incrementally rolling out machines with a new puppet role
Date: 2013-03-24 22:52:19

Last week https://twitter.com/jasonneylon[Jason] and I with (a lot of) help from https://twitter.com/timrgoodwin[Tim] have been working on moving several of our applications from https://twitter.com/jasonneylon[Passenger] to https://github.com/blog/517-unicorn[Unicorn] and decided that the easiest way to do this was to create a new set of nodes with this setup.

The architecture we're working with looks like this at a VM level:

image::{{<siteurl>}}/uploads/2013/03/architecture.png[Architecture,600]

The 'nginx LB' nodes are responsible for routing all the requests to their appropriate application servers and the 'web' nodes serve the different applications initially using Passenger.

We started off by creating a new 'nginx LB' node which we pointed to a new 'web ELB' and just put one 'unicorn web' node behind it so that we could test everything was working.

We then pointed 'www.uswitch.com' at the IP of our new 'nginx LB' node in our +++<cite>+++/etc/hosts+++</cite>+++ file and checked that the main flows through the various applications were working correctly.

Once we were happy this was working correctly we increased the number of 'unicorn web' nodes to three and then repeated our previous checks while tailing the log files across the three machines to make sure everything was ok.

The next step was to send some of the real traffic to the new nodes and check whether they were able to handle it.

Initially we thought that we could put our 'unicorn web' nodes alongside the 'web' nodes but we realised that we'd made some changes on our new 'nginx LB' nodes which meant that the 'unicorn web' nodes needed to receive requests proxies through there rather than from the old style node.

A combination of Jason and https://twitter.com/siddharthdawara[Sid] came up with the idea of just plugging our new 'nginx LB' into the 'nginx ELB' and having the processing of the whole request treated separately.

Our intermediate architecture therefore looked like this:

image::{{<siteurl>}}/uploads/2013/03/arhictecture-rollover.png[Arhictecture rollover,600]

We initially served 1/4 of the requests from the Unicorn and watched the performance of the nodes via http://newrelic.com/[New Relic] to check that everything was working expected.

One thing we did notice was that the CPU usage on the Unicorn nodes was really high because we'd set up each Unicorn process with 5 workers which meant that we had 25 workers on the VM in total. In comparison our Passenger instances used 5 workers in total.

Once we'd sorted that out we removed one of the 'nginx LB' nodes from the 'nginx ELB' and served 1/3 of the traffic from our new stack.

We didn't see any problems so we removed all the 'nginx LB' nodes and served all the traffic from our new stack for half an hour.

Again we didn't notice any problems so our next step before we can decommission the old nodes is to run the new stack for a day and iron out any problems before using it for real.
Treat servers as cattle: Spin them up, tear them down
Date: 2013-04-27 14:22:10

A few agos I wrote a post about http://www.markhneedham.com/blog/2013/04/07/treating-servers-as-cattle-not-as-pets/[treating servers as cattle, not as pets] in which I described an approach to managing virtual machines at uSwitch whereby we frequently spin up new ones and delete the existing ones.

I've worked on teams previously where we've also talked about this mentality but ended up not doing it because it was difficult, usually for one of two reasons:

* *Slow spin up* - this might be due to the cloud providers infrastructure, doing too much on spin up or I'm sure a variety of other reasons.
* *Manual steps involved in spin up* - the process isn't 100% automated so we have to do some manual tweaks. Once the machine is finally working we don't want to have to go through that again.

Martin Fowler http://martinfowler.com/bliki/FrequencyReducesDifficulty.html[wrote a post a couple of years ago] where he said the following:

____
One of my favorite soundbites is: *if it hurts, do it more often*. It has the happy property of seeming nonsensical on the surface, but yielding some valuable meaning when you dig deeper
____

I think it applies in this context too and I have noticed that the more frequently we tear down and spin up new nodes the easier it becomes to do so.

Part of this is because there's been less time for http://www.markhneedham.com/blog/2013/04/27/puppet-package-versions-to-pin-or-not-to-pin/[changes to have happened in package repositories] but we are also *more inclined to optimise things that we have to do frequently* so the whole process is faster as well.

For example in one of our sets of machines we need to give one machine a specific tag so that when the application is deployed it sets up a bunch of cron jobs to run each evening.

Initially this was done manually and we were quite reluctant to ever tear down that machine but we've now got it all automated and it's not a big deal anymore - it can be cattle just like the rest of them!

One neat rule of thumb https://twitter.com/philandstuff[Phil] taught me is that if we make major changes to our infrastructure we should spin up some new machines to check that it still actually works.

If we don't do this then when we actually need to spin up a new node because of a traffic spike or machine corruption problem it's not going to work and we're going to have to fix things in a much more stressful context.

For example we recently moved some repositories around in github and although it's a fairly simple change spinning up new nodes helped us see all the places where we'd failed to make the appropriate change.

While I appreciate taking this approach is more time consuming in the short term I'd argue that if we automate as much of the pain as possible in the long run it will probably be beneficial.