markhneedham/blog huggingface dataset:
Fabric/Boto: boto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['QuerySignatureV2AuthHandler'] Check your credentials
Date: 2013-01-15 00:37:01

In our Fabric code we make use of https://github.com/boto/boto[Boto] to connect to the EC2 API and pull back various bits of information and the first time anyone tries to use it they end up with the following stack trace:

[source,text]
----

  File "/Library/Python/2.7/site-packages/fabric/main.py", line 717, in main
    *args, **kwargs
  File "/Library/Python/2.7/site-packages/fabric/tasks.py", line 332, in execute
    results['<local-only>'] = task.run(*args, **new_kwargs)
  File "/Library/Python/2.7/site-packages/fabric/tasks.py", line 112, in run
    return self.wrapped(*args, **kwargs)
  File "/Users/mark/projects/forward-puppet/ec2.py", line 131, in running
    instances = instances_by_zones(running_instances(region, role_name))
  File "/Users/mark/projects/forward-puppet/ec2.py", line 19, in running_instances
    ec2conn = ec2.connect_to_region(region)
  File "/Library/Python/2.7/site-packages/boto/ec2/__init__.py", line 57, in connect_to_region
    for region in regions(**kw_params):
  File "/Library/Python/2.7/site-packages/boto/ec2/__init__.py", line 39, in regions
    c = EC2Connection(**kw_params)
  File "/Library/Python/2.7/site-packages/boto/ec2/connection.py", line 94, in __init__
    validate_certs=validate_certs)
  File "/Library/Python/2.7/site-packages/boto/connection.py", line 936, in __init__
    validate_certs=validate_certs)
  File "/Library/Python/2.7/site-packages/boto/connection.py", line 548, in __init__
    host, config, self.provider, self._required_auth_capability())
  File "/Library/Python/2.7/site-packages/boto/auth.py", line 633, in get_auth_handler
    'Check your credentials' % (len(names), str(names)))
boto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['QuerySignatureV2AuthHandler'] Check your credentials
----

We http://stackoverflow.com/questions/5396932/why-are-no-amazon-s3-authentication-handlers-ready[haven't told Boto about our AWS credentials] and I've come across two ways of providing them:

== As environment variables

[source,text]
----

export AWS_ACCESS_KEY_ID="aws_access_key_id"
export AWS_SECRET_ACCESS_KEY="aws_secret_access_key"
----

== In the file +++<cite>+++~/.boto+++</cite>+++

[source,text]
----

[Credentials]
aws_access_key_id = aws_access_key_id
aws_secret_access_key = aws_secret_access_key
----

And that should do the trick!
Fabric: Tailing log files on multiple machines
Date: 2013-01-15 00:20:49

We wanted to tail one of the log files simultaneously on 12 servers this afternoon to try and see if a particular event was being logged and rather than opening 12 SSH sessions decided to get http://docs.fabfile.org/en/1.5/[Fabric] to help us out.

My initial attempt to do this was the following:

[source,text]
----

fab -H host1,host2,host3 -- tail -f /var/www/awesome/current/log/production.log
----

It works but the problem is that by default Fabric runs the specified command http://docs.fabfile.org/en/1.4.4/usage/execution.html#execution-strategy[one machine after the other] so we've actually managed to block Fabric with the tail command on 'host1'.

The output of host1's log file will be printed to the terminal but nothing from the other two hosts.

http://junctionbox.ca/[Nathan] showed me how to get around this problem by making use of Fabric's parallel execution which we can enable with the '-P' option:

[source,text]
----

fab -P --linewise -H host1,host2,host3 -- tail -f /var/www/awesome/current/log/production.log
----

We also used the 'likewise' flag to ensure that data between the different tail processes didn't get mixed up although this wasn't necessary because Fabric http://docs.fabfile.org/en/1.4.4/usage/parallel.html[defaults to likewise if you're using parallel execution] mode anyway.

On a side-note, http://oobaloo.co.uk/kafka-for-uswitchs-event-pipeline[Paul Ingles wrote up the approach taken to make data from log files more accessible] using a Kafka driven event pipeline but in this case we haven't got round to wiring this data up yet so Fabric it is for now.
Ego Depletion
Date: 2013-06-04 23:16:29

On the recommendation of https://twitter.com/michael_jones[Mike Jones] I've been reading through Daniel Kahneman's http://www.amazon.co.uk/Thinking-Fast-Slow-Daniel-Kahneman/dp/0141033576/ref=sr_1_1?ie=UTF8&qid=1370274373&sr=8-1&keywords=thinking+fast+and+slow[Thinking Fast and Slow] in which the first part of the book covers our two styles of thinking:

* *System 1* - operates automatically and quickly, with little or no effort and no sense of voluntary control.
* *System 2* - allocates attention to the effortful mental activities that demand it, including complex computations. The operations of System 2 are often associated with the subjective experience of agency, choice, and concentration.

He then uses a series of stories to explain this in more detail but I found an experiment run by Roy Baumeister the most interesting:

____
A series of surprising experiments [\...] has shown conclusively that all variants of voluntary effort - cognitive, emotional, or physical - draw at least partly on a shared pool of mental energy. Their experiments involve successive rather than simultaneous tasks. Baumeister's group has repeatedly found that an effort of will or self control is tiring; if you have had to force yourself to do something, you are less willing or less able to exert self-control when the next challenge comes around. The phenomenon has been named *ego depletion*.
____

This particularly resonates with me as I've frequently seen people (including myself) let a series of events involving another person to go by before finally snapping over something seemingly innocuous.

Quite frequently the other person had no idea that the way they were behaving was irritating so the reaction comes as a surprise to them!

We can get around this problem to some extent by http://www.markhneedham.com/blog/2013/05/23/feedback-reacting-immediately/[providing timely feedback] but in order for that to happen I think we need to admit to ourselves when we are frustrated rather than pretending that it doesn't bother us.

It will probably feel silly to address these innocuous events so early on when we don't think they're bothering us that much but I think it's better than the explosive alternative!
neo4j/cypher: CREATE with optional properties
Date: 2013-06-20 06:31:11

I've written before about using the http://docs.neo4j.org/chunked/milestone/query-create.html[cypher CREATE statement] to http://www.markhneedham.com/blog/2013/05/25/neo4jcypher-properties-or-relationships-its-easy-to-switch/[add inferred information to a neo4j graph] and sometimes we want to do that but have to deal with optional properties while creating our new relationships.

For example let's say we have the following people in our graph with the 'started' and 'left' properties representing their tenure at a company:

[source,cypher]
----

CREATE (person1 { personId: 1, started: 1361708546 })
CREATE (person2 { personId: 2, started: 1361708546, left: 1371708646 })
CREATE (company { companyId: 1 })
----

We want to create a 'TENURE' link from them to the company including the 'started' and 'left' properties when applicable and might start with the following query:

[source,cypher]
----

START person = node:node_auto_index('personId:1 personId:2'),
      company = node:node_auto_index('companyId:1')
CREATE person-[:TENURE_AT { started: person.started, left: person.left }]-company
RETURN person, company
----

which throws an exception because not all people have a 'left' property:

[source,text]
----

Error: org.neo4j.cypher.EntityNotFoundException: The property 'left' does not exist on Node[1]
----

We tweak our query a bit to make the property optional:

[source,cypher]
----

START person = node:node_auto_index('personId:1 personId:2'),
      company = node:node_auto_index('companyId:1')
CREATE person-[:TENURE_AT { started: person.started, left: person.left? }]-company
RETURN person, company
----

which still doesn't work:

[source,text]
----

Error: java.lang.IllegalArgumentException: Null parameter, key=left, value=null
----

After looking at this for a while https://twitter.com/apcj[Alistair] pointed out that we should just split the updating of the optional property from the creation of the relationship so we end up with the following:

[source,cypher]
----

START person = node:node_auto_index('personId:1 personId:2'),
      company = node:node_auto_index('companyId:1')
CREATE person-[tenure:TENURE_AT { started: person.started }]-company
WITH person, tenure, company
WHERE HAS(person.left)
SET tenure.left = person.left
RETURN person, company
----

The http://console.neo4j.org/?id=gdbpuv[code is on the console] if you want to see how it works.
neo4j: WrappingNeoServerBootstrapper and the case of the /webadmin 404
Date: 2013-06-19 05:32:50

When people first use neo4j they frequently start out by http://docs.neo4j.org/chunked/stable/tutorials-java-embedded.html[embedding it in a Java application] but eventually they want to explore the graph in a more visual way.

One simple way to do this is to start neo4j in server mode and use the web console.

Our initial code might read like this:

[source,java]
----

public class GraphMeUp {
    public static void main(String[] args) {
        GraphDatabaseService graphDb = new EmbeddedGraphDatabase("/path/to/data/graph.db");
    }
}
----

or:

[source,java]
----

public class GraphMeUp {
    public static void main(String[] args) {
        GraphDatabaseService graphDb = new GraphDatabaseFactory().
          newEmbeddedDatabaseBuilder("/path/to/data/graph.db").
          newGraphDatabase();
    }
}
----

And to start our graph up in server mode we can use the +++<cite>+++http://components.neo4j.org/neo4j-server/1.9/apidocs/org/neo4j/server/WrappingNeoServerBootstrapper.html[WrappingNeoServerBootstrapper]+++</cite>+++ class which is packaged in neo4j-server so we first need to add that dependency:

[source,xml]
----

 <dependency>
    <groupId>org.neo4j.app</groupId>
    <artifactId>neo4j-server</artifactId>
    <version>1.9</version>
</dependency>
----

[source,java]
----

public class GraphMeUp {
    public static void main(String[] args) {
        GraphDatabaseService graphDb = new GraphDatabaseFactory().
          newEmbeddedDatabaseBuilder("/path/to/data/graph.db").
          newGraphDatabase();

        new WrappingNeoServerBootstrapper((GraphDatabaseAPI)graphDb).start();
    }
}
----

If we then browse to http://localhost:7474/webadmin/ we'll be greeted by a 404 error:

[source,text]
----

HTTP ERROR 404

Problem accessing /webadmin/. Reason:

    Not Found
Powered by Jetty://
----

sad panda :(

Until I came across http://stackoverflow.com/questions/8111959/use-wrappingneoserverbootstrapper-with-spring-data-neo4j[this post on StackOverflow by Michael] I didn't realise that there's actually another dependency that we need to include to get the web admin goodness!

To get things worked as we'd expect we need to include the following dependency: ~~~xml +++<dependency>++++++<groupId>+++org.neo4j.app+++</groupId>+++ +++<artifactId>+++neo4j-server+++</artifactId>+++ +++<version>+++1.9+++</version>+++ +++<type>+++jar+++</type>+++ +++<classifier>+++static-web+++</classifier>+++ +++<scope>+++compile+++</scope>++++++</dependency>+++ ~~~

I hadn't come across the +++<cite>+++classifier+++</cite>+++ attribute before but what this does is include the following JAR: ~~~bash $ ls -alh ~/.m2/repository/org/neo4j/app/neo4j-server/1.9/neo4j-server-1.9-static-web.jar -rw-r--r-- 1 markhneedham staff 3.5M 17 Jun 11:28 /Users/markhneedham/.m2/repository/org/neo4j/app/neo4j-server/1.9/neo4j-server-1.9-static-web.jar ~~~

If we run our code again we should see the bright and cheery web admin interface and all is good with the world.
Unix/awk: Extracting substring using a regular expression with capture groups
Date: 2013-06-26 15:23:14

A couple of years ago I http://www.markhneedham.com/blog/2011/09/12/gawk-getting-story-numbers-from-git-commit-messages/[wrote a blog post] explaining how I'd used http://www.gnu.org/software/gawk/[GNU awk] to extract story numbers from git commit messages and I wanted to do a similar thing today to extract some node ids from a file.

My eventual solution looked like this:

[source,bash]
----

$ echo "mark #1000" | gawk '{ match($0, /#([0-9]+)/, arr); if(arr[1] != "") print arr[1] }'
1000
----

But in the comments an alternative approach was suggested which used the Mac version of awk and the RSTART and RLENGTH global variables which get set when a match is found:

[source,bash]
----

$ echo "mark #1000" | awk 'match($0, /#[0-9]+/) { print substr( $0, RSTART, RLENGTH )}'
#1000
----

Unfortunately http://stackoverflow.com/questions/2957684/awk-access-captured-group-from-line-pattern[Mac awk doesn't seem to capture groups] so as you can see it includes the # character which we don't actually want.

In this instance it wasn't such a big deal but it was more annoying for the node id extraction that I was trying to do:

[source,bash]
----

$ head -n 5 log.txt
Command[27716, Node[7825340,used=true,rel=14547348,prop=31734662]]
Command[27716, Node[7825341,used=true,rel=14547349,prop=31734665]]
Command[27716, Node[7825342,used=true,rel=14547350,prop=31734668]]
Command[27716, Node[7825343,used=true,rel=14547351,prop=31734671]]
----

[source,bash]
----

$ head -n 5 log.txt | awk 'match($0, /Node\[([^,]+)/) { print substr( $0, RSTART, RLENGTH )}'
Node[7825340
Node[7825341
Node[7825342
Node[7825343
Node[7825336
----

I ended up having to +++<cite>+++brew install gawk+++</cite>+++ and using a variation of the +++<cite>+++gawk+++</cite>+++ command I mentioned at the beginning of this post:

[source,bash]
----

$ head -n 5 log.txt | gawk 'match($0, /Node\[([^,]+)/, arr) { print arr[1]}'
7825340
7825341
7825342
7825343
7825336
----
The Affect Heuristic
Date: 2013-06-06 22:36:04

In my continued reading of Daniel Kahneman's http://www.amazon.co.uk/Thinking-Fast-Slow-Daniel-Kahneman/dp/0141033576/ref=sr_1_1?ie=UTF8&qid=1370274373&sr=8-1&keywords=thinking+fast+and+slow[Thinking Fast and Slow] I've reached the section which talks about *the affect heuristic* which seems particularly applicable to the technical decisions that we make.

____
The dominance of conclusions over arguments is most pronounced where emotions are involved. The psychologist Paul Slovic has proposed an affect heuristic in which *people let their likes and dislikes determine their beliefs about the world*.
____

The way I've seen this heuristic coming into play in the software world is when we do an 'objective' overview of the technical tools/options that we could use to solve a particular problem.

We may do this by coming up with a list of advantages/disadvantages for each technology but the way we come up with this will probably be influenced by which of the technologies we prefer.

We'll therefore place strong emphasis on the advantages of a technology and not think too much of disadvantages or work arounds that we have to implement.

For example if Clojure were the technology in question then as an advocate of Clojure you might focus on the reduced lines of code and benefits of the functional way of programming and place less emphasis on the learning curve that new team members will have to overcome.

Equally if you weren't a fan of Clojure then you'd do the opposite.

I covered similar ground in http://www.markhneedham.com/blog/2013/02/28/compatible-opinions-confirmation-bias/[a post I wrote a few months ago] about compatible opinions where I suggested people used confirmation bias to back up their own opinions.

I think the affect heuristic is slightly different though because it applies even when we think we're being impartial in our judgement.

When I read things I like to try and think what action I should be taking as a result of learning new information. In this case I think the take away is to be more self aware than usual when talking about things we're passionate about.

One way to achieve that could be to run our opinions via someone who is knowledgeable in the subject area but is less emotionally involved.

It'd be interesting to see whether this resonates with others as well and how you handle it.
Java: Finding/Setting JDK/$JAVA_HOME on Mac OS X
Date: 2013-06-15 10:28:28

As long as I've been using a Mac I always understood that if you needed to set +++<cite>+++$JAVA_HOME+++</cite>+++ for any program, it should be set to +++<cite>+++/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK+++</cite>+++.

On my machine this points to the 1.6 JDK:

[source,bash]
----

$ ls -alh /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK
/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK -> /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents
----

This was a bit surprising to me since I've actually got Java 7 installed on the machine as well so I'd assumed the symlink would have been changed:

[source,bash]
----

$ java -version
java version "1.7.0_09"
Java(TM) SE Runtime Environment (build 1.7.0_09-b05)
Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode)
----

https://twitter.com/andres_taylor[Andres] and I were looking at something around this yesterday and wanted to set +++<cite>+++$JAVA_HOME+++</cite>+++ to the location of the 1.7 JDK on the system if it had been installed.

We eventually came across http://developer.apple.com/library/mac/#qa/qa1170/_index.html[the following article] which explains that you can use the +++<cite>+++/usr/libexec/java_home+++</cite>+++ command line tool to do this.

For example, if we want to find where the 1.7 JDK is we could run the following:

[source,bash]
----

$ /usr/libexec/java_home -v 1.7
/Library/Java/JavaVirtualMachines/jdk1.7.0_09.jdk/Contents/Home
----

And if we want 1.6 the following does the trick:

[source,bash]
----

$ /usr/libexec/java_home -v 1.6
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
----

We can also list all the JVMs installed on the machine:

[source,bash]
----

$ /usr/libexec/java_home  -V
Matching Java Virtual Machines (3):
    1.7.0_09, x86_64:	"Java SE 7"	/Library/Java/JavaVirtualMachines/jdk1.7.0_09.jdk/Contents/Home
    1.6.0_45-b06-451, x86_64:	"Java SE 6"	/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
    1.6.0_45-b06-451, i386:	"Java SE 6"	/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home

/Library/Java/JavaVirtualMachines/jdk1.7.0_09.jdk/Contents/Home
----

I'm not sure how I've never come across this command before but it seems pretty neat.
neo4j/cypher/Lucene: Dealing with special characters
Date: 2013-06-15 09:53:15

neo4j uses http://lucene.apache.org/core/[Lucene] to handle indexing of nodes and relationships in the graph but something that can be a bit confusing at first is how to handle special characters in Lucene queries.

For example let's say we set up a database with the following data:

[source,cypher]
----

CREATE ({name: "-one"})
CREATE ({name: "-two"})
CREATE ({name: "-three"})
CREATE ({name: "four"})
----

And for whatever reason we only wanted to return the nodes that begin with a hyphen.

A hyphen is a special character in Lucene so if we forget to escape it we'll end up with an impressive stack trace:

[source,cypher]
----

START p = node:node_auto_index("name:-*") RETURN p;
----

[source,text]
----

==> RuntimeException: org.apache.lucene.queryParser.ParseException: Cannot parse 'name:-*': Encountered " "-" "- "" at line 1, column 5.
==> Was expecting one of:
==>     <BAREOPER> ...
==>     "(" ...
==>     "*" ...
==>     <QUOTED> ...
==>     <TERM> ...
==>     <PREFIXTERM> ...
==>     <WILDTERM> ...
==>     "[" ...
==>     "{" ...
==>     <NUMBER> ...
==>
----

So we change our query to escape the hyphen:

[source,cypher]
----

START p = node:node_auto_index("name:\-*") RETURN p;
----

which results in the following exception:

[source,text]
----

==> SyntaxException: invalid escape sequence
==>
==> Think we should have better error message here? Help us by sending this query to cypher@neo4j.org.
==>
==> Thank you, the Neo4j Team.
==>
==> "START p = node:node_auto_index("name:\-*") RETURN p"
==>
----

The problem is that the cypher parser also treats '\' as an escape character so we need to use two of them to make our query do what we want:

[source,cypher]
----

START p = node:node_auto_index("name:\\-*") RETURN p;
----

[source,text]
----

==> +------------------------+
==> | p                      |
==> +------------------------+
==> | Node[4]{name:"-one"}   |
==> | Node[5]{name:"-two"}   |
==> | Node[6]{name:"-three"} |
==> +------------------------+
==> 3 rows
----

Alternatively, as https://twitter.com/cleishm[Chris] pointed out, we could make use of parameters in which case we don't need to worry about how the cypher parser handles escaping:

[source,ruby]
----

require 'neography'

neo = Neography::Rest.new
query = "START p = node:node_auto_index({query}) RETURN p"
result = neo.execute_query(query, { :query => 'name:\-*'})

p result["data"].map { |x| x[0]["data"] }
----

[source,text]
----

$ bundle exec ruby params.rb
[{"name"=>"-one"}, {"name"=>"-two"}, {"name"=>"-three"}]
----
Supporting production code: Start with the simple things
Date: 2013-12-20 18:07:36

A few months ago I wrote about http://www.markhneedham.com/blog/2013/07/22/lessons-from-supporting-production-code/[my experiences supporting production code] while working at http://www.uswitch.com/[uSwitch].

Since then I've been working on support for Neo4j customers and I've realised that there are a couple of other things to keep in mind while debugging production problems that I missed from the initial list.

== Keep a clear head / Hold back your assumptions

The first is that it's very helpful to completely *clear your head of any assumptions* when looking at a problem.

I've got into the habit of pattern matching different error messages that I come across with root causes and while that's sometimes useful, often there are subtle differences which mean the root cause is different.

Although I still sometimes fall into the assumptions trap I've found that it helps to ask exactly what someone is trying to do rather than immediately trying to solve the problem.

== Look for the simple things

Along with the assumptions another mistake I make is to imagine the most complicated version of events that could lead to a problem manifesting.

Sometimes this is the case but more frequently a configuration setting may have been misunderstood or a query poorly designed and the problem can be resolved more easily.

To stop myself making this mistake I have a rough flow chart in my head working down from simpler causes to more complicated ones for different problem areas.

As I said, I still do make assumptions and look for complicated reasons for problems but by keeping these two things in mind I think/hope I'm doing it less often than I used to!