markhneedham/blog huggingface dataset:
Clojure: Checking for a nil value in a collection
Date: 2009-11-21 22:11:22

Something which I wanted to do recently was write a function that would indicate whether a collection contained a nil value.

I initially incorrectly thought the 'http://clojure.org/api#toc170[contains?]' function was the one that I wanted:

[source,lisp]
----

(contains? '(1 nil 2 3) nil)
=> false
----

I thought it would work the same as the Java equivalent but that function actually checks whether a key exists in a collection rather than a value. It's more useful when dealing with maps.

There's http://groups.google.com/group/clojure/browse_frm/thread/49173d05a6781f62/47084ba4eec07c26?lnk=gst&q=contains%3F#47084ba4eec07c26[more discussion on the consistency of the API on the mailing list].

Luckily the documentation guides us towards the 'http://clojure.org/api#toc523[some]' function:

My first attempt was to write an anonymous function to check if there was a 'nil' in the list:

[source,lisp]
----

(some #(= % nil) '(1 nil 2 3))
=> true
----

[source,lisp]
----

(some #(= % nil) '(1  2 3))
=> nil
----

http://twitter.com/fogus/status/5904916921[fogus showed me an even better way] by making use of the built in 'http://clojure.org/api#toc388[nil?]' function:

[source,lisp]
----

(some nil? '(1 nil 2 3))
----

Another approach would be to make use of the Java 'contains' method as http://twitter.com/philip_schwarz[Philip Schwarz] http://twitter.com/philip_schwarz/statuses/5905264987[pointed out]:

[source,lisp]
----

(.contains '(1 nil 2 3) nil)
=> true
----

I noticed that when you use Java methods in Clojure with collections then the result will either be 'true' or 'false' whereas when you use Clojure built in functions then it's more likely to be 'true' or 'nil'.

I guess this is linked to the idea that http://www.markhneedham.com/blog/2009/11/20/clojure-a-few-things-ive-been-tripping-up-on/['nil' is false in Clojure] so it doesn't make much difference what the return value is.

When I'm using a language I've got into the habit of just trying out the API in the way that I expect it to work rather than paying a lot of attention to what the API documentation says.

I think this is something I'll need to work out to avoid much frustration!
Knowing when to persevere and when to change approach
Date: 2009-11-08 09:57:41

It strikes me that one of the most important skills to develop in software development is knowing when to keep going with an approach to a problem and when we should stop and try something else.

This situation doesn't always happen because if we have two people available and realise before we start on the task that there is some doubt as to which solution is the most appropriate then we can adopt a http://www.markhneedham.com/blog/2009/09/19/set-based-concurrent-engineering-a-simple-example/[set based approach] whereby we try out multiple potential solutions in parallel.

However I've found that there are times when we might think that one solution is much better than the alternatives and we will start working on that and discard the alternatives for the moment.

Frequently the approach we choose will solve our problem but on other occasions we can end up getting a bit bogged down when it doesn't work out as expected.

At this stage I often find that my instinct is to try and solve these problems and while I think this is certainly a valid approach it's also very easy to end up http://www.markhneedham.com/blog/2008/10/25/dont-shave-the-yak-ask-why-are-we-doing-this/[shaving the yak].

This seems to happen much more frequently when working alone and even when we're getting feedback which suggests that what we're trying isn't working we can be reluctant to back track.

I think there is some pride involved as I frequently find myself really wanting to make the current approach work as I believe that a better developer than myself would be able to do so.

Sometimes that is the case but more often than not when I work through the problem with a colleague their suggestion is to try another approach rather than continuing to struggle with the current one.

It doesn't make a lot of sense to spend all our time going down one avenue when our goal is solve a particular problem and the solution is a means to that end.

When discussing this with http://pilchardfriendly.wordpress.com/[Nick] he pointed out that we also need to recognise when we should persevere with the approach.

One way to avoid too much yak shaving is to set a http://en.wikipedia.org/wiki/Timeboxing[timebox] by when we should either have a better idea of whether we are going to be able to solve the problem with this solution or if we need to consider an alternative.

If there's some light at the end of the tunnel with an approach after this period then I would be inclined to keep on going but it's sometimes difficult to tell how close we actually are and how much is wishful thinking!

I still sometimes find myself struggling to decide whether to keep going or change direction so it'd be interesting to know if anyone has ideas around doing so more effectively.
TDD: Useful when new on a project
Date: 2009-11-06 21:57:10

Something which I've noticed over the last few projects that I've worked on is that at the beginning when I don't know very much at all about the code base, domain and so on is that pairing with someone to TDD something seems to make it significantly easier for me to follow what's going on than other approaches I've seen.

I thought that it was probably because I'm more used to that approach than any other but in Michael Feathers' description of TDD in 'http://www.amazon.com/gp/product/0131177052?ie=UTF8&tag=marneesblo-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0131177052[Working Effectively With Legacy Code]' he points out the following:

____
One of the most valuable things about TDD is that it *lets us concentrate on one thing at a time*. We are either writing code or refactoring, we are never doing both at once.
____

It also temporarily removes us from the feeling of drowning in all the new information which is typical at the start of projects. In a way it reminds me of the 'http://apprenticeship.oreilly.com/wiki/retreat_into_competence[Retreat Into Competence]' pattern from 'http://www.amazon.com/gp/product/0596518382?ie=UTF8&tag=marneesblo-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0596518382[Apprenticeship Patterns]' as it gives us a chance to regain composure and take in all the new information.

We can probably make a much more useful contribution if we only need to understand one small piece of the system rather than having to understand everything which can often be the case with what Feathers coins the edit & pray approach to development.

It's still necessary to spend some time trawling around the code base working out how everything fits together but I'm now more aware that it's also really useful to take some time to just focus on one much smaller class or piece of functionality.

Perhaps also something to keep in mind the next time I'm pairing with someone who's new to a project that I've been working on for a while.
Requirements: The story points focus
Date: 2009-11-23 11:46:52

Something which an agile approach on a project typically gives us is the ability to change requirements rapidly based on the different types of feedback we typically get over the course of the project.

One way that we can lose this advantage is by getting caught up by the number of story points being completed and using this as the measure of success.

The flexibility to change has an impact on the number of story points that may be completed in a given iteration - if we start doing some work on a story and then get feedback from the business while it is still in progress it's possible that we will end up with more work to do than we had previously.

Once a story is estimated http://www.markhneedham.com/blog/2009/02/11/agile-re-estimating-cards/[we don't typically go back and change that estimate] unless there was a change in the assumptions being made before we play it. This means that any changes we make based on feedback would not result in re-estimation of the story card.

We now have more work to do without our measurement mechanism recognising that.

It gets even worse if we start working on a feature and then get feedback from the business that the feature is no longer useful to them and we should just stop working on it.

At this point a fairly common response is to 'lock down' the requirements so that we'll only work on the currently defined requirements and no new ones or changes will be permitted.

While this might make some logical sense it misses the point that we're supposed to try and deliver http://www.dancingmango.com/blog/2009/11/18/about-a-successful-project-that-was-a-failiure/[something that is actually useful rather than something which just meets the initial requirements].

In reality one of two things, or perhaps a combination of both, ends up happening:

* There will still be changes and all we've done is create the illusion that there won't be any.
* The business will now push to get exactly the features that were specified originally even if the development team ends up spending a disproportionate amount of time creating those features for the value they actually provide.

Story points are really useful for allowing us to get an idea of the relative size of stories and can be helpful for getting a rough understanding of the amount of work we can complete in a given time period but we need to be careful how we use them.

Our goal at the end of the day is to deliver working software.

Story points are just a mechanism for giving us some feedback on how well we are doing that. They're not the goal in itself.
A reminder to talk to the rubber duck
Date: 2009-11-15 21:06:42

Alongside http://www.markhneedham.com/blog/2009/05/26/the-value-of-a-fresh-mind/[taking a break from it] perhaps one of the most effective ways to solve a tricky problem is to describe it to someone else.

== When pairing

This typically isn't a problem when pair programming although it can still happen if a pair stays together too long and both start making the same possibly incorrect assumptions when trying to solve a problem.

In this case it makes sense to call someone else over who can lend a fresh perspective to the problem.

At this stage all three people can work through the problem together and then the third person can go back to whatever they were doing before or this can be the catalyst for a pair rotation whereby one of the original two will drop from the pair.

== When working alone

When working alone I get into the situation where something 'should' be working but isn't much more frequently. I find it really frustrating when this happens so I probably call a colleague over to look through the problem more frequently than most.

Frequently just describing the problem to them so that they can help is enough to realise where the problem lies.

I've been working with http://twitter.com/raphscallion[Raph] recently and last week on a couple of occasions I found myself imagining a conversation with Raph about a problem I was having and I found that just doing that was enough for me to see where I'd gone wrong.

Luciano Passuello refers to this to as http://litemind.com/solve-your-problems-simply-by-saying-them-out-loud/[talking to the rubber duck] and he describes a way that we can apply this technique silently as well:

____
There are situations when you can't make noises or simply can't afford the social awkwardness of it ("`excuse me while I talk to my duck...`"). Fear not: do your rubber ducking in writing instead -- sure, you won't have the benefits of verbalization, but the technique still works fine. One way to do it that works particularly well is to write an e-mail message explaining your problem. Pretend you're going to send it to the smartest (and busiest) person in the world. Describe the problem in detail, the solutions you tried so far and why they didn't work. List your assumptions and make sure you include all relevant information. Be both thorough and objective. When you are done writing the message, you'll probably have had many ideas to try out. If not, find a specialized forum and just send it as it is: with your perfectly-crafted message, I'm sure the Internet gods will help you.
____

I actually hadn't come across this blog post when I accidentally came across this idea while writing an email describing why something I'd been working on wasn't quite working.

As I described the main part of the problem it became obvious from reading it back to myself that I'd missed out a small but important detail.

I was then able to go back to the code and fix the problem!
Mercurial: hg bisect
Date: 2009-11-14 11:20:13

We've been using http://mercurial.selenic.com/wiki/[Mercurial] locally on the project I've been working on and http://fragmental.tw/[Phil] showed me a cool feature called 'http://mercurial.selenic.com/wiki/BisectExtension[bisect]' a couple of weeks ago which can be helpful for working out which revision we managed to break our code in.

It's been ported across from http://git-scm.com/[Git] and is included in Mercurial from version 1.0.0 rather than just being an extension.

From the bisect extension page:

____
Its behaviour is fairly simple: it takes a first revision known to be correct (i.e. without the bug) and a last revision known to be bad (i.e. with the bug). The bisect extension ouputs a revision halfway between the good and the bad ones and lets you test it. If this revision is a good one, you mark it as good with hg bisect good, otherwise you mark it as bad with hg bisect bad. In both cases, bisect outputs a new revision to test, halfway between the good and the bad ones. You repeat until only one revision is left: the culprit.
____

The usage has changed a bit now that it's included as part of the initial download.

I was working on something yesterday and checking in fairly regularly before realising that I'd broken something.

I was fairly sure that the break had happened in the tip (revision 98) but I could only remember it definitely working in revision 96.

I defined the good and bad revisions like this:

[source,text]
----



~~~text


In this case revision 97 was now checked out and I checked that the code was working with that revision before marking it:


~~~text


It's now able to work out that the problem is in fact in revision 98:


~~~text

The first bad revision is:
changeset:   98:86260809c309
tag:         tip
user:        mneedham
date:        Fri Nov 13 16:31:02 2009 +1100
summary:     seem to have screwed up the graphs. Not sure how
----

I managed to mess up setting up the good and bad revisions a few times - I somehow had the impression that I needed to manually update to the original good and bad revisions which isn't the case.

The reset command was my friend before I worked out what I was doing wrong:

[source,text]
----

hg bisect -r
----

It seems like a neat little feature. I'm sure there are lots of other cool things like this in Mercurial so if you know any let me know!

* Update *
As Rob points out in the comments the command is 'bisect' rather than 'bisec' as I had it originally. Both commands work for the moment though!
TDD: Combining the when and then steps
Date: 2009-11-14 00:17:57

I've written before about http://www.markhneedham.com/blog/2009/04/13/tdd-balancing-dryness-and-readability/[my favoured approach of writing tests in such a way that they have clear 'Given/When/Then' sections] and something which I come across quite frequently is tests where the latter steps have been combined into one method call which takes care of both of these.

An example of this which I came across recently was roughly like this:

[source,java]
----

@Test
public void shouldCalculatePercentageDifferences() {
	verifyPercentage(50, 100, 100);
	verifyPercentage(100, 100, 0);
	verifyPercentage(100, 50, -50);
}
----

[source,java]
----

private void verifyPercentage(int originalValue, int newValue, int expectedValue) {
	assertEquals(expectedValue, new PercentageCalculator().calculatePercentage(originalValue, newValue));
}
----

This code is certainly adhering to the DRY principle although it took us quite a while to work out what the different numbers being passed into 'verifyPercentage' were supposed to represent.

With this type of test I think it makes more sense to have a bit of duplication to make it easier for us to understand the test.

We changed this test to have its assertions inline and make use of the http://code.google.com/p/hamcrest/[Hamcrest] library to do those assertions:

[source,java]
----

@Test
public void shouldCalculatePercentageDifferences() {
	assertThat(new PercentageCalculator().calculatePercentage(50, 100), is(100));
	assertThat(new PercentageCalculator().calculatePercentage(100, 100), is(0));
	assertThat(new PercentageCalculator().calculatePercentage(100, 50), is(-50));
}
----

I think we may have also created a field to instantiate 'PercentageCalculator' so that we didn't have to instantiate that three times.

Although we end up writing more code than in the first example I don't think it's a problem because it's now easier to understand and we'll be able to resolve any failures more quickly than we were able to previously.

As Michael Feathers points out during Jay Fields' 'http://blog.jayfields.com/2009/06/developer-testing-welcome-to-beta-test.html[Beta Test]' presentation we need to remember why we try and adhere to the http://en.wikipedia.org/wiki/Don%27t_repeat_yourself[DRY principle] in the first place.

To paraphrase his comments:

____
In production code if we don't adhere to the DRY principle then we might make a change to a piece of code and we won't know if there's another place where we need to make a change as well. In test code the tests always tell us where we need to make changes because the tests will break.
____
Pair Programming/Helping/Working Collaboratively
Date: 2009-11-22 16:43:24

Dan North has been presenting his 'http://www.infoq.com/presentations/north-pimp-my-architecture[Pimp my architecture]' talk again at QCon San Francisco this week and after reading the http://twitter.com/IT_Voyager/statuses/5838886106[hugely] http://twitter.com/adolfosousa/statuses/5840052174[positive] http://twitter.com/guilhermecaelum/statuses/5840219363[feedback] http://twitter.com/fabiokung/statuses/5840379985[on Twitter] I decided to watch some of it again.

The idea of getting people to help each other rather than pair program is what stood out for me this time, something which http://twitter.com/bguthrie/status/5839065698[Brian Guthrie also pointed out]:

____
"We didn't do pairing, we did 'helping'. You can't get alpha progs to 'pair' but they'll tell you what they know."
____

It's been quite common at the places that I've worked at for pair programming to be frowned upon and sometimes that's not the most important battle to fight so the team will be more selective about its use.

I was discussing this with some colleagues this week and it does seem that pair programming is a trigger word/phrase with negative connotations for productivity.

On the other hand, something which is rarely frowned upon is having a team 'working collaboratively' and in fact most of the time this is considered a very good thing.

When we work alone I still find myself http://www.markhneedham.com/blog/2009/11/15/a-reminder-to-talk-to-the-rubber-duck/[calling a colleague over to help me out when I get stuck] and while I could probably work something out alone given enough time I don't think that approach makes sense when working on it with someone else leads to us solving the problem much more quickly.

Josh Bloch speaks along similar lines in the interview with him in http://www.amazon.com/gp/product/1430219483?ie=UTF8&tag=marneesblo-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1430219483[Coders at Work]:

____
I don't like working in total isolation. When I'm writing a program and I come to a tricky design decision, I just have to bounce it off someone else. At every place I've worked, I've had one or more colleagues I could bounce ideas off of. That's critically important for me; I need that feedback. \... I've known people who don't feel this way - who are willing to program in a vacuum. I think it hurts them.
____

I think it's fairly inevitable that if a team is to deliver software effectively then there are going to be times when people are working together to solve problems.

What we choose to call that can be varied depending on the context we find ourself in.
Clojure: The 'apply' function
Date: 2009-11-25 11:59:11

In my continued playing around with Clojure I came across the 'http://clojure.org/api#toc86[apply]' function which is used when we want to call another function with a number of arguments but have actually been given a single argument which contains the argument list.

The example that I've been trying to understand is applying 'http://clojure.org/api#toc537[str]' to a collection of values.

I started off with the following:

[source,lisp]
----

(str [1 2 3])
=> "[1 2 3]"
----

This just returns the string representation of the vector that we passed it, but what we actually want is to get an output of "123".

The 'apply' function allows us to do that:

[source,lisp]
----

(apply str [1 2 3])
=> "123"
----

That is semantically/conceptually the same as doing this:

[source,lisp]
----

(str 1 2 3)
=> "123"
----

I didn't quite understand how that could work though and my assumption was that somewhere in the Clojure source the above function call would be happening.

The definition of 'apply' is as follows:

[source,lisp]
----

(defn apply
  "Applies fn f to the argument list formed by prepending args to argseq."
  {:arglists '([f args* argseq])}
  [#^clojure.lang.IFn f & args]
    (. f (applyTo (spread args))))
----

The first thing which I hadn't realised is that when you have an '&' before a parameter definition then any arguments provided will be put into a list.

If we break down the example above we end up with the following:

[source,lisp]
----

(. str (applyTo (spread [[1 2 3]])))
----

The 'spread' function is defined like so:

[source,lisp]
----

(defn spread
  {:private true}
  [arglist]
  (cond
   (nil? arglist) nil
   (nil? (next arglist)) (seq (first arglist))
   :else (cons (first arglist) (spread (next arglist)))))
----

In this case we only have one item in 'arglist' so on line 6 the 'next arglist' expression evaluates to nil.

This means that we create a seq from the first argument of the 'arglist' which is '[1 2 3]'.

Working our way back up to the 'apply' function what we end up with is this:

[source,lisp]
----

(. str (applyTo (seq [1 2 3])))
=> "123"
----

This calls through to an 'applyTo' method defined on the 'clojure.lang.IFn' interface.

I'm not sure which of the implementations 'str' maps to but it seems like the 'str' function would eventually be called from the Java code with each of the values in the sequence passed in as a separate argument which is pretty neat!