markhneedham/blog huggingface dataset:
Cruise: Pipelining for fast visual feedback
Date: 2009-01-19 21:38:20

One of the cool features in build servers like http://studios.thoughtworks.com/cruise-continuous-integration[Cruise] and http://www.jetbrains.com/teamcity/[Team City] is the ability to create build http://studios.thoughtworks.com/cruise-continuous-integration/deployment-pipelines[pipelines].

I have done a bit of work using this feature in previous projects but the key driver for doing so there was to create a chain of producers/consumers (producing and consuming artifacts) eventually resulting in a manual step to put the application into a testing environment.

While this is certainly a good reason to create a build pipeline, http://twitter.com/davcamer[a colleague] pointed out an equally useful way of using this feature to split the build into separate steps pipelined together.

By doing this we get a nice graphical display from the cruise dashboard which allows us to see where the build is failing, therefore pointing out where we need to direct our focus.

image::{{<siteurl>}}/uploads/2009/01/pipeline-activity-small.png[pipeline_activity_small.png,324]

One way to use the pipelines is to work out the distinct potential areas where you would want to signal that something needs to be investigated and then make each of these targets a separate build target.

For example we could set it up like so:

No dependency build \=> +
Services build  \=> +
End to End smoke test build \=> +
Full build

== Benefits of this approach

The benefit of this approach is that it helps to *create more confidence in the build process*.

When we have a long running build it is easy to get into a state where it is failing after 3/4 of the build has run and all we get is the red failed build to indicate something has gone wrong.  We can drill down to find out where the failure is but it's not as obvious.

The approach we have taken to checking in is that it is fine to do as long as the first stage of the build is green. This has worked reasonably well so far and failure further down stream has been fixed relatively quickly.

== Things to watch for

We have setup the final step of the build to be a manual step due to the fact that it takes quite a long time to run and we've been unable to get a dedicated machine to run an agent on. Ideally we would have it running constantly on its own agent.

This isn't run as frequently as when we had it running automatically and I guess the danger is that we are *pushing problems further down stream* rather than catching them early. Hopefully this issue will be solved if we can get a dedicated agent running this build.

We're still looking for ways to improve our build process but this is what's currently working for reasonably well for us at the moment. It would be interesting to hear what others are doing.
F# vs C# vs Java: Functional Collection Parameters
Date: 2009-01-19 19:24:25

I wrote a post about a month ago on using http://www.markhneedham.com/blog/2008/12/17/functional-collection-parameters-in-c/[functional collection parameters] in C# and over the weekend http://fabiopereira.me/blog/[Fabio] and I decided to try and contrast the way you would do this in Java, C# and then F# just for fun.

== Map

Map evaluates a high order function on all the elements in a collection and then returns a new collection containing the results of the function evaluation.

Given the numbers 1-5, return the square of each number

Java

[source,java]
----

int[] numbers = { 1,2,3,4,5};
for (int number : numbers) {
    System.out.println(f(number));
}

private int f(int value) {
    return value*value;
}
----

C#

[source,csharp]
----

new List<int> (new[] {1, 2, 3, 4, 5}.Select(x => x*x)).ForEach(Console.WriteLine);
----

F#

[source,text]
----

[1..5] |> List.map (fun x -> x*x) |> List.iter (printfn "%d");;
----

== Filter

Filter applies a predicate against all of the elements in a collection and then returns a collection of elements which matched the predicate.

Given the numbers 1-5, print out only the numbers greater than 3:

Java

[source,java]
----

int[] numbers = { 1,2,3,4,5};
for (int number : numbers) {
    f(number);
}

private void f(int value) {
    if(value > 3) {
        System.out.println(value);
    }
}
----

C#

[source,csharp]
----

new List<int> { 1,2,3,4,5}.FindAll(x => x > 3).ForEach(Console.WriteLine);
----

F#

[source,text]
----

[1..5] |> List.filter (fun x -> x > 3) |> List.iter (printfn "%d");;
----

== Reduce

Reduce applies a high order function against all the elements in a collection and then returns a single result.

Given a list of numbers 1-5, add them all together and print out the answer

Java

[source,java]
----

int sum = 0;
int[] numbers = { 1,2,3,4,5};
for (int number : numbers) {
    sum += number;
}
System.out.println(sum);
----

C#

[source,java]
----

Console.WriteLine(new[] {1, 2, 3, 4, 5}.Aggregate(0, (accumulator, x) => accumulator + x));
----

F#

[source,text]
----

[1..5] |> List.fold_left (+) 0 |> printfn "%d";;
----

== In Summary

I was surprised that we could achieve these results in relatively few lines of Java. The C# and F# versions are still more concise but the Java version isn't too bad. The http://commons.apache.org/collections/api-2.1.1/org/apache/commons/collections/CollectionUtils.html#select(java.util.Collection,%20org.apache.commons.collections.Predicate)[Apache Commons Library has a class] which allows you to write these in a functional way but the need to use anonymous methods means it's not as clean as what you can achieve in C# and F#.

I think there is still a bit of a mindset switch to make from thinking procedurally about these things to thinking in a way that allows you to make the most of functional programming concepts.

Keeping the code as declarative as possible and reducing the amount of state in our code are the most obvious things I have learned so far from playing with F#.
C#: Builder pattern still useful for test data
Date: 2009-01-21 23:49:13

I had thought that with the ability to use the new http://davidhayden.com/blog/dave/archive/2006/12/04/ObjectInitializationExpressions.aspx[object initalizer] syntax in C# 3.0 meant that the builder pattern was now no longer necessary but some recent refactoring efforts have made me believe otherwise.

My original thought was that the builder pattern was really useful for providing a nicely chained way of creating objects, but after a bit of discussion with some colleagues I have come across three different reasons why we might want to use the builder pattern to create test data:

* It creates a nice to read fluent interface describing the object being created. This argument holds more for Java rather than C# where we now have object initializers.
* Domain objects are a bit complicated to create - encapsulate this logic in the builder.
* We want to default non null data on some of the fields in our object. If we don't explicitly set a value for a property in C# it defaults to null.

Even with the object initializer syntax we can still end up having to specify extra data that we don't really care about in our test. The following is not uncommon:

[source,csharp]
----

new Foo {Bar = "bar", Baz = "baz", Bling = "bling"};
----

[source,csharp]
----

public class Foo
{
    public string Bar {get; set;}
    public string Baz { get; set; }
    public string Bling { get; set; }
}
----

Let's say we only care about Bar for this test though but Baz and Bling are both being used in our code so we end up with a Null Reference Exception if we don't set values for them. We can quickly end up having this redundant data being repeated across all our tests.

In steps the builder pattern!

[source,csharp]
----

new FooBuilder().Bar("bar").Build();
----

[source,csharp]
----

public class FooBuilder
{
    private string bar = "defaultBar";
    private string baz = "defaultBaz";
    private string bling = "defaultBling";

    public FooBuilder Bar(string value)
    {
        bar = value;
        return this;
    }

    public FooBuilder Baz(string value)
    {
        baz = value;
        return this;
    }

    public FooBuilder Bling(string value)
    {
        bling = value;
        return this;
    }

    public Foo Build()
    {
        return new Foo {Bar = bar, Baz = baz, Bling = bling};
    }
}
----

It takes a bit more code to setup but every time we use the builder it saves us typing in extra data that we don't need.

It would be even better if we could not have to call that 'Build' method and we can get around this by using the http://realfiction.net/?q=node/138[implicit operator], the problem being that you need to apply it to the target class (i.e. Foo) rather than the class you want to implicitly convert from (i.e. FooBuilder).

I don't really want to change a production code class just for test purposes so the 'Build' will have to stay in there for the time being.
Javascript Dates - Be aware of mutability
Date: 2009-01-07 23:17:05

It seems that http://www.markhneedham.com/blog/2008/09/18/using-javautildate-safely/[much like in Java], dates in Javascript are mutable, meaning that it is possible to change a date after it has been created.

We had this painfully shown to us when using the http://datejs.com/[datejs] library to http://www.markhneedham.com/blog/2009/01/07/javascript-add-a-month-to-a-date/[manipulate some dates].

The erroneous code was similar to this:

[source,javascript]
----

var jan312009 = new Date(2008, 1-1, 31);
var oneMonthFromJan312009 = new Date(jan312009.add(1).month());
----

See the subtle error? Outputting these two values gives the following:

[source,text]
----

Fri Feb 29 2008 00:00:00 GMT+1100 (EST)
Fri Feb 29 2008 00:00:00 GMT+1100 (EST)
----

The error is around how we have created the 'oneMonthFromJan312009':

____
var oneMonthFromJan312009 = new Date(*jan312009.add(1).month()*);
____

We created a new Date but we are also changing the value in 'jan312009' as well.

It was the case of having the bracket in the wrong place. It should actually be after the 'jan312009' rather than at the end of the statement.

This is the code we wanted:

[source,javascript]
----

var jan312009 = new Date(2008, 1-1, 31);
var oneMonthFromJan312009 = new Date(jan312009).add(1).month());
----

Which leads to more expected results:

[source,text]
----

Sat Jan 31 2009 00:00:00 GMT+1100 (EST)
Sat Feb 28 2009 00:00:00 GMT+1100 (EST)
----
Javascript: Add a month to a date
Date: 2009-01-07 23:00:58

We've been doing a bit of date manipulation in Javascript on my current project and one of the things that we wanted to do is add 1 month to a given date.

We can kind of achieve this using the standard date libraries but it doesn't work for edge cases.

For example, say we want to add one month to January 31st 2009. We would expect one month from this date to be February 28th 2009:

[source,javascript]
----

var jan312009 = new Date(2009, 1-1, 31);
var oneMonthFromJan312009 = new Date(new Date(jan312009).setMonth(jan312009.getMonth()+1));
----

The output of these two variables is:

[source,javascript]
----

Sat Jan 31 2009 00:00:00 GMT+1100 (EST)
Tue Mar 03 2009 00:00:00 GMT+1100 (EST)
----

Not quite what we want!

Luckily there is a library called http://www.datejs.com/[datejs] which has taken care of this problem for us. It provides a really nice http://www.martinfowler.com/bliki/DomainSpecificLanguage.html[DSL] which makes it very easy for us to do what we want.

We can add a month to a date very easily now:

[source,javascript]
----

var jan312009 = new Date(2009, 1-1, 31);
var oneMonthFromJan312009 = new Date(jan312009).add(1).month();
----

[source,javascript]
----

Sat Jan 31 2009 00:00:00 GMT+1100 (EST)
Sat Feb 28 2009 00:00:00 GMT+1100 (EST)
----

There are loads of other useful date manipulation functions which you can read more about on the http://code.google.com/p/datejs/wiki/APIDocumentation[API], just don't forget that date in Javascript is mutable so any manipulation done to dates contained in vars will change the original value.
Coding Dojo #8: Isola
Date: 2009-01-30 11:17:58

Our latest coding dojo involved writing the board game http://en.wikipedia.org/wiki/Isola_(board_game)[Isola] in Java.

== The Format

We used the http://codingdojo.org/cgi-bin/wiki.pl?RandoriKata[Randori] approach again with around 8 or 9 people participating for the majority of the session, our biggest turnout yet. I think the majority of people had the opportunity to drive a couple of times over the evening.

We had the pair driving at the front of the room and everyone else further back to stop the tendency of observers to whiteboard stuff.

== What We Learnt

* We took the approach of trying to get a useable front end for the game working as quickly as possible this time which was a bit different to our normal approach where we tend to focus more heavily on the modeling side of the problem. We therefore decided to do the *simplest thing that could possibly work* and hardcoded the representation of the board as a string, which surprisingly (to me at least) proved adequate for the whole time we were coding and it was only towards the end that we felt we needed to put in a more robust data structure. Certainly a lesson for me of the value of not over engineering a solution.
* This led to a discussion around what sort of situation this would represent on a real project. The closest we came up with was that of using an in memory repository early on until a real database is actually needed. We need to *trade off the complexity we are adding in by doing this integration versus the gains we get from integrating early*.
* An cool approach which http://pilchardfriendly.blogspot.com/[Nick] showed us was to *always implement code inline*, make the test pass and then extract it into methods or classes as part of a refactoring step. This is similar to the idea of http://www.thekua.com/atwork/2008/10/sprouting-inner-classes/[sprouting inner classes] which Pat Kua wrote about a couple of months ago. This takes the idea of taking small steps even further which can only be a good thing.
* It was interesting to see that without a domain expert we ending up changing the ubiquitous language in the code quite frequently but never really came up with one that made sense to everyone - when talking about the code we ended up with people doing translation between their understanding of various concepts.

== For next time

* The plan for next week is to continue working on Isola - it turned out to be quite an interesting game to try and model, simple enough that we could understand the rules quickly but complicated enough that it takes a while to implement.
Coding Dojo #5: Uno
Date: 2009-01-08 23:41:57

We ran our 5th coding dojo on Thursday night, writing the card game http://en.wikipedia.org/wiki/Uno_(game)[Uno] in Java. We didn't all know the rules so http://www.youtube.com/watch?v=biNXzYOxmb8[this video] explained it - surely a parody but you never know!

== The Format

We used the http://codingdojo.org/cgi-bin/wiki.pl?RandoriKata[Randori] approach again with 6 people participating for the majority of the session. Everyone paired with everyone else at least once and sometimes a couple of times.

We had the pair driving at the front of the room and everyone else further back to stop the tendency of observers to whiteboard stuff.

== What We Learnt

* *Modeling games is really good for practicing design skills*. Most people had played the game so we had domain experts who could use their knowledge to help drive out the API of the various classes. We didn't get to the scoring part of the game in the time available but it was quite cool to see our code with all the terms detailed in Wikipedia's entry on the term.
* We managed to *drive the design much more effectively* than we have done on previous sessions. The flexibility to move between classes depending on where it made most sense to test from next was finally there and we didn't end up with the problem we've had on previous sessions where we ended up with coarsely grained tests and then tried to code the whole application in one go.
* It was quite painful for me personally having to manually perform operations on collections in Java rather than having the selection of http://www.markhneedham.com/blog/2008/12/17/functional-collection-parameters-in-c/[functional operators] that are available in C# 3.0.
* It wasn't a new learning but I've noticed in my project work that I've become a lot more keen to *keep the steps really small* - there is a bit of pressure on you to do this in a dojo situation and I think it's just continued over from there. Every time I try to be too clever and take a big step something inevitably doesn't work and I end up doing the small steps anyway. It's also a lot of fun coding in this type of environment and watching how others approach problems and how they pair with each other. If you get a chance to attend a dojo I think it'd definitely be worthwhile.

== Other Dojo Thoughts

* Some ideas for future coding dojos that we discussed were:
 ** Concurrency - using the http://code.google.com/p/retlang/[Retlang]/http://code.google.com/p/jetlang/[Jetlang] libraries
 ** Do some stuff with http://code.google.com/p/webdriver/[Web Driver]
 ** Modeling games
 ** Taking an open source project and refactoring it
* I notice there are a couple of sessions of coding/coding dojos planned for Jason Gorman's http://parlezuml.com/softwarecraftsmanship/sessions/programme.htm[Software Craftsmanship conference]. It will be interesting to see how those work out, especially if there are high numbers of participants. We've always had a fairly small number of people involved which I think has helped to keep everyone involved. I'm not convinced it would be effective with many more participants.
jQuery datepicker IE6 positioning bug
Date: 2009-01-06 21:57:06

We've been using the http://docs.jquery.com/UI/Datepicker[jQuery datepicker] on my current project and came across some strange behaviour with regards to the positioning of the calendar in IE6.

The calendar was always positioning itself right at the top of the screen instead of just below the textbox it was hooked up to but in Firefox it was working fine.

After a bit of exploration in the jQuery code (http://ui.jquery.com/latest/ui/ui.datepicker.js[ui.datepicker.js]) we worked out that the 'document.documentElement.clientHeight' call in the '_checkOffset' function was always returning a value of 0 meaning that the position of the calendar was always right at the top of the screen.

[source,text]
----

_checkOffset: function(inst, offset, isFixed) {
		var pos = inst.input ? this._findPos(inst.input[0]) : null;
		var browserWidth = window.innerWidth || document.documentElement.clientWidth;
		var browserHeight = window.innerHeight || document.documentElement.clientHeight;
...
----

It turned that we were missing the doctype at the top of the HTML page which make the page standards compliant and as a result document.documentElement.clientHeight returns the proper height.

[source,text]
----

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
----

Not having this set puts IE into Quirks Mode meaning that document.documentElement.clientHeight  always returns 0.

We found a post on the Webmaster World Forum which helps explain http://www.webmasterworld.com/forum21/11096.htm[why this change makes a difference].
jQuery: Approaches to testing
Date: 2009-01-24 09:36:32

We've been doing a bit of work with http://jquery.com/[jQuery] and true to our TDD roots we've been trying to work out the best way to test drive our coding in this area.

There seem to be 3 main ways that you can go about doing this, regardless of the testing framework you choose to you. We are using http://github.com/nkallen/screw-unit/tree/master[screw-unit] for our javascript testing.

== Mock everything out

The idea here is that we mock out all calls made to jQuery functions and then we assert that the expected calls were made in our test.

Taking this approach means we are able to reduce the dependencies in our tests and they run very quickly.

The problem is that a lot of assertions become assertions checking that certain operations were called on the DOM since jQuery makes a lot of these type of calls in its code. The tests therefore end up being quite long and difficult to understand unless you were the one who initially wrote them.

We also effectively end up testing how the framework interacts with the DOM rather than testing our own code with this approach.

== Don't mock anything

The opposite approach is to just test directly against jQuery and then do assertions against the part of the DOM affected by the javascript code we are testing.

The problem here is that if you want to test against plugins which make ajax requests or carry out animation effects then the tests become dependent on how long these calls take and we need to find a way to block the test until those calls return which is quite difficult!

== Only stub certain calls

The happy medium is that we test directly against the jQuery library but stub out ajax requests and animation effects. Our test assertions are against the state of the DOM to check that it was changed in the way that we expected.

This approach allows us to test our javascript code in terms of its behaviour without testing the internals of how jQuery works.

The wiring up of events to different controls on the page is done in our test but the actual logic that happens when these events are fired is in our js file under test.

We currently favour this last approach as it seems to give us the best of both worlds so to speak. It be would be interesting to hear how are other people going about Javascript testing.
VMware: Accessing host server
Date: 2009-06-02 21:36:46

I've been doing all my spare time .NET development from within VMWare for about the last year or so and now and then it's quite useful to be able to access the host machine either to get some files from there or to access a server that's running on the host.

The former problem is solved by going to 'Virtual Machines \-> Shared Folders' and clicking on the + button on the bottom left of the menu to add a folder that you want to share.

This folder will be accessible by going to  'My Network Places \-> Entire Network \-> VMWare Shared Folders \-> .host \-> Shared Folders' from Windows Explorer or by typing '\.host\Shared Folders' into the Windows Explorer address bar.

The latter is something I'd not wanted to do until today when I wanted to access a CouchDB server I had running via http://janl.github.com/couchdbx/[CouchDBX] (thanks to http://twitter.com/jchris/statuses/1996608145[J Chris Anderson for the recommendation]) from a .NET application that I was running inside VMWare.

From the host environment I can view all the databases in CouchDB by going to 'http://127.0.0.1:5984/_utils' but from VMWare I need to make use of the Gateway IP address which can be found by typing 'ipconfig' at the command prompt inside the VM.

The database listing is now available at 'http://the.gateway.ip:5984/_utils'.