markhneedham/blog huggingface dataset:
I don't have time not to test!
Date: 2009-04-18 09:25:17

I recently read a blog post by Joshua Lockwood where he spoke of some people who claim they http://www.lostechies.com/blogs/joshua_lockwood/archive/2009/04/13/quot-i-don-t-have-time-to-test-quot.aspx[don't have time to test].

Learning the TDD approach to writing code has been one of best things that I've learnt over the last few years - before I worked at ThoughtWorks I didn't know how to do it and the only way I could verify whether something worked was to load up the application and manually check it.

It was severely painful and on one particular occasion I managed to put some code with a bug into production because I didn't know all the places that making that code change would impact.

It's not a good way of working and I'm glad I've been given the opportunity to work with people who have showed me a better way.

My experience pretty much matches a comment made by http://www.lostechies.com/members/chrismissal/default.aspx[Chris Missal] on the post where he pointed out that you are going to test your code anyway so you might as well automate that test!

____
"You're already testing with the debugger, TestPage1.aspx, or whatever\... Just save that code and automate it!"
____

I've just spent the last 2 hours doing some http://www.markhneedham.com/blog/2009/04/18/f-refactoring-that-little-twitter-application-into-objects/[refactoring on an F# twitter application I'm working on] and because I didn't write any tests it's been a very painful experience indeed.

Every time I make a change I have to copy all the code into F# interactive, run the code and then manually make sure that I haven't broken anything.

I've been doing this in fairly small steps - make one change then run it - but *the cycle time is still much greater than it would be if I had just put some tests around the code in the first place*.

I think we should be looking to test more than just the 'complex code' as well - there have been numerous occasions when I've put the logic for a conditional statement the wrong way around and a test has come to the rescue.

It pretty much applies to all the languages that I've worked in and if we can't see how to easily create an automated test for a bit of code then http://www.markhneedham.com/blog/2008/11/30/tdd-if-its-hard-to-test-reflect-on-your-approach/[it's a sign that we're doing something wrong] and we might want to take a look at that!
Pair Programming: The Code Fairy
Date: 2009-04-10 19:28:18

One of the hardest situations that comes up  when pair programming is when you want to solve a problem in a certain way but you can't persuade your pair that it's the approach you should take.

The temptation in these situations is to wait until your pair isn't around, maybe by staying late at the end of the day or coming in early the next day and then making the changes to the code that you wanted to make but didn't when you were pairing with them.

My colleague http://camswords.wordpress.com/[Cam Swords] has coined this pattern of behaviour '*the code fairy*' and it's something that we want to try and avoid in software development teams.

The thing to note here is that you're following another direction than you had been when pairing, not that you're working alone on the code just because you're pair can't work with you at the moment for example.

The problem with following this approach is that you *lose the benefit of pairing* in that you have two eyes looking at the code and reducing the likelihood of stupid code being put into the code base - by stupid I mean code which has been written with cleverness in mind rather than simplicity. I find that when I work alone those types of ideas are much more likely to seem a good idea than when I have someone else there to put me right.

When we can't convince our pair to follow our approach it is often due to the fact that we can't articulate why we prefer it well enough to convince them. In a recent conversation with Cam about the problem of persuading people of our approach he pointed out that even if we are really good at constructing an argument for why our approach is better our pair may still not want to do it and we can't do a lot about it.

Most recently I've been really enthused by the idea of writing http://blog.objectmentor.com/articles/2009/03/22/the-sucessor-value-pattern[functional OO code] but I find it very frustrating that I can't currently clearly explain why I prefer this approach to writing code over an approach which favours there being more mutable state.

I really want to go and change the code to follow that approach but it would be inconsistent with the rest of the code base as well as being less clear to my team mates since it would have been me who wrote it alone rather than doing it in collaboration with a colleague.

Another problem is that since we ended up going and re-writing the code ourselves the same situation will happen again in the future so *we haven't really achieved anything apart from temporarily making the code a bit better in our eyes*. In addition there are now less people on the team who understand that particular bit of code which is not a good position to be in when we're working in a team.

It's definitely not easy to resist the urge to go and 'fix' the code on our own but for the benefit of our team mates it's something we need to consider.
DDD: Making implicit concepts explicit
Date: 2009-04-23 12:36:25

One of my favourite parts of the http://domaindrivendesign.org[Domain Driven Design] book is where Eric Evans talks about making implicit concepts in our domain model explicit.

The book describes this process like so:

____
Many transformations of domain models and the corresponding code happen when developers recognize a concept that has been hinted at in discussion or present implicitly in the design, and they then represent it explicitly in the model with one or more objects or relationships.
____

Lu and I were working on a small application to parse the WCF message log file on our project into more readable chunks whereby each request in the file would be outputted into another file so that it would be possible to read them individually.

We decided to create a little domain model for this since the code seemed to be getting a bit tricky to handle when it was all being written inline inside a main method.

To start with we just had a collection of requests which was an accurate representation of the way that the data was being stored in the log file.

We collected all these requests and then created individual files for each of them. We also grouped these request files under directories by the session that the request was from.

The input/output of our application looked a bit like this:

image::{{<siteurl>}}/uploads/2009/04/implicitexplicit.gif[implicitexplicit.gif,442]

The next idea suggested for this little application was that it would be cool if we could put the characters 'FAIL' into the file name of any requests which failed and also into the folder name of any sessions which had failing requests inside them.

We tried to do this with our original model but everything we did resulted in adding more and more code to the Request object which didn't seem to belong to it. The tipping point for me was when we ended up with Request.SessionFolderName as a property.

Eventually we realised that what had been implicit would now need to be made explicit and the Session came to be an object in our domain model.

What I found most interesting about this process was that we were always talking about the Session but it didn't actually exist in our model!

The model in our code now pretty much represents the format in which we out outputting the data and with Session as an explicit concept it makes it much easier to make changes in the future.
F#: Not equal/Not operator
Date: 2009-04-25 22:12:43

While continuing playing with http://www.markhneedham.com/blog/2009/04/18/f-refactoring-that-little-twitter-application-into-objects/[my F# twitter application] I was trying to work out how to exclude the tweets that I posted from the list that gets displayed.

I actually originally had the logic the wrong way round so that it was only showing my tweets!

[source,ocaml]
----

let excludeSelf (statuses:seq<TwitterStatus>) =
    statuses |> Seq.filter (fun eachStatus ->  eachStatus.User.ScreenName.Equals("markhneedham"))
----

Coming from the world of Java and C# '!' would be the operator to find the screen names that don't match my own name. So I tried that.

[source,ocaml]
----

let excludeSelf (statuses:seq<TwitterStatus>) =
    statuses |> Seq.filter (fun eachStatus -> !(eachStatus.User.ScreenName.Equals("markhneedham")))
----

Which leads to the error:

[source,text]
----

Type constraint mismatch. The type 'bool' is not compatible with the type 'bool ref'.
----

If we look at the definition of '!' we see the following:

[source,ocaml]
----

(!);;

> val it : ('a ref -> 'a)
----

It's not a logical negation operator at all. In actual fact it's an operator used to deference a mutable reference cell.

What I was looking for was actually http://stackoverflow.com/questions/239888/logical-negation-operator-in-f-equivalent[the 'not' operator].

[source,ocaml]
----

let excludeSelf (statuses:seq<TwitterStatus>) =
    statuses |> Seq.filter (fun eachStatus ->  not (eachStatus.User.ScreenName.Equals("markhneedham")))
----

I could also have used the '<>' operator although that would have changed the implementation slightly:

[source,ocaml]
----

let excludeSelf (statuses:seq<TwitterStatus>) =
    statuses |> Seq.filter (fun eachStatus ->  eachStatus.User.ScreenName <> "markhneedham")
----

The http://research.microsoft.com/en-us/um/cambridge/projects/fsharp/manual/FSharp.Core/Microsoft.FSharp.Core.Operators.html[Microsoft Research F# website] seems to be quite a useful reference for understanding the different operators.
Pair Programming: When your pair steps away
Date: 2009-05-03 19:08:27

I've been having a bit of a discussion recently with some of my colleagues about what we should do when pair programming and one of the people in the pair has to step away to go and help someone else or to take part in an estimation session or whatever it happens to be.

If we're pairing in an effective way then it should be possible for the person still at the computer to keep on going on the story/task that the pair were working on alone. Obviously sometimes that isn't the case especially if http://www.markhneedham.com/blog/2008/11/05/pair-programming-the-over-eager-driver/[one person has been driving for the majority of the time] but for this post we'll assume that both people are capable of continuing alone.

Continuing alone doesn't necessarily mean that you become the http://www.markhneedham.com/blog/2009/04/10/pair-programming-the-code-fairy/[code fairy], which is where one of the people in a pair goes and implements the functionality of something they had been pairing on in their own favoured style.

My initial thought is that if the absence is only short term then you shouldn't plow on too much otherwise you need to spend time bringing them back on the same page when they return.

To give an example, a couple of weeks ago I was pairing with a colleague and we were retrieving a value from a Dictionary if it existed and creating a value in the Dictionary if it did not exist.

http://www.twitter.com/davcamer[Dave] had recently shown me quite a clean way of doing this which I wanted to discuss with my colleague in case they hadn't seen it before - the approach we had been taking to solve this problem wasn't along these lines before my pair was called away.

[source,csharp]
----

public class DictionaryExample
{
    private readonly Dictionary<string, string> values = new Dictionary<string, string>();

    public string FindValue(string key)
    {
        if(!values.ContainsKey(key))
        {
            values[key] = "somethingNew";
        }
        return values[key];
    }
}
----

When he came back I suggested this approach and he was happy to go with it.

I sometimes write down stuff I'm unsure of when pairing and I find that if my pair goes off for a short amount of time then this can be a useful time to look that up.

If we decide to keep on going during their absence then I think it's important that we keep going down the same path  that we were when we were pairing to reduce the amount of catching up our pair needs to do when they return.

If they have gone away for a longer period of time then we should treat it as them having left the pair and we can look for someone else to pair with or just code as if we were working alone.

That's my current thinking on this - some colleagues have suggested they think it's better if we just keep on coding regardless but I think this approach finds a happy medium.
Adding humour to Tester/Developer collaboration
Date: 2009-05-04 23:43:03

Pat Kua has a http://www.thekua.com/atwork/2009/04/improving-collaboration-between-developers-and-testers/[recent post where he talks about the language used between testers and developers] when talking about defects that testers come across when testing some functionality and while I would agree with him that the language used is important, I've always found that injecting some humour into the situation takes the edge off.

As http://dahliabock.wordpress.com/2009/05/03/its-our-code/[Dahlia points out I think this is probably only possible if there is good rapport between the developers and testers on the team] so perhaps this has been the case for the teams I've worked on.

I would find it quite disappointing if my first attempt at a story cleared all the way through to business sign off without a tester in the team at least coming up with some cases where it doesn't work properly - I try to think of the scenarios that someone with a testing hat on would come up with but they are way better at that role than I am so there's bound to be something that I've missed.

Now this doesn't mean that I should keep recreating the same types of defects/bugs over and over again - that would be the http://www.leanmap.com/index.php?categoryid=74[waste of re-learning] and doesn't add a whole lot of value.

In all the teams I've worked on there has definitely been a bit of banter between the testers and the developers whereby the testers tell us off 'tongue in cheek' for putting so many bugs into the code and we respond by asking them not to keep on breaking the application.

I've always felt that this approach worked reasonably well although it should probably be pointed out that I only do that with my ThoughtWorks colleagues where we pretty much have an implicit understanding that we are not criticising each other when talking in such a (supposedly) blunt manner.

If there's any underlying lesson from this approach then I would suggest it's that developers would be better of assuming that a tester is probably going to find a bug in their code and that they shouldn't assume something is finished just because it is development complete.

Testers on the other hand maybe can be less confrontational (as Pat suggests) when they find bugs - the developers didn't put them in there deliberately! You guys just happen to be way better at using the application in a way that finds its' flaws than we are.

Keeping it light hearted is also way more fun!
F#: Entry point of an application
Date: 2009-05-02 01:56:09

In an attempt to see whether or not the http://www.markhneedham.com/blog/2009/05/02/f-erlang-style-messaging-passing/[mailboxes I've been working on for my twitter application] were actually processing messages on different threads I ran into the problem of defining the entry point of an F# application.

I thought it would be as simple as defining a function called 'main' but I put this function into my code ran the executable and nothing happened!

Googling the problem a bit led me to believe that it is possible to do but that the function needs to be the last thing that happens in the compilation sequence of the project. i.e.  it needs to be on the last line of the last file that gets compiled.

That just seemed wrong to me and a http://stackoverflow.com/questions/663894?sort=newest[Stack Overflow thread] suggested that it should be possible to get around this problem http://lorgonblog.spaces.live.com/blog/cns!701679AD17B6D310!412.entry[by using the EntryPointAttribute on the main function].

I tried this leading to the following code:

[source,ocaml]
----

[<EntryPoint>]
let main args =
    printfn "in main function"
    0
----

It seems like if you have the EntryPointAttribute on a function that function needs to be of type 'string array \-> int' hence the returning of 0 by this function.

This still didn't solve my problem though and when I tried to build the project it was now failing to compile but no errors were showing up on the Visual Studio error list.

This gave me the chance to try out an F# build tool I came across a couple of weeks ago called http://code.google.com/p/fake/[Fake].

I setup a build file just to compile the code based on http://www.navision-blog.de/2009/04/01/getting-started-with-fake-a-f-sharp-make-tool/[Steffen Forkman's blog post] and then tried to compile the project, leading to the following error:

[source,text]
----

error FS0191: A function labelled with the 'EntryPointAttribute' atribute must be the last declaration in the last file in the compilation sequence.
----

Which suggests that it doesn't actually matter whether or not you have the attribute or not, the main method still needs to be the last step of compilation.

I wanted to get something working so I've just rearranged my fsproj file so that the file with the main function in is the last one listed but it seems a ridiculous fix and I'm sure there must be a better way to do this.

Any ideas?
Tackling the risk early on at a task level
Date: 2009-05-11 23:54:12

I http://www.markhneedham.com/blog/2008/11/10/agile-putting-the-risk-up-front/[wrote previously about the idea of tackling the risky tasks in a project early on] - an idea that I learnt about when reading Alistair Cockburn's http://www.markhneedham.com/blog/2008/11/05/crystal-clear-book-review/[Crystal Clear].

Towards the end of the post I wondered whether we could apply this idea at a story level whereby we would identify the potentially risky parts of a story and make sure that we addressed those risks before they became problematic to us.

I define risk in this sense to mean something that we don't know a lot about and therefore don't know how long it is going to take, something which we think might be difficult to do or something which is likely to cause us problems.

From my experience I've noticed that there tends to be risk around any boundaries with other systems which are likely to be a black box to us and in areas which we lack information about the best approach and therefore need to do some research/spiking first.

I've been trying to apply this approach and in a recent example my pair and I needed to fix a bug whereby some values on the website weren't being correctly updated when the user changed a value in a particular field.

We started by informally working out the different changes we might need to make in order to fix this problem:

* Javascript to handle the page refreshing with the new values
* C# mapping data from service into JSON object
* Service call to get back the new values

We had quite a tight timeframe to make this fix and it seemed clear that the call to the service was probably the biggest area of risk in fixing this bug - we had done some investigation which indicated that we hadn't been sending a particular piece of data to the service and we weren't sure what would happen when we did.

As it turned out when we did pass this data we weren't quite getting the response that we expected but we were able to communicate with the other team and get the problem resolved really quickly.

The only thing I would have changed about our approach to this problem was that we tested whether or not the service was working by making a call to it through the UI having ensured that we were passing the correct data through to it.

We would have been able to find out whether our integration was working much more quickly if we had just written an automated test directly hitting the service and made some assertions on the results. This only became apparent to me while watching an http://railsconf.blip.tv/file/2089545/[Uncle Bob presentation which included a section about the value of testing].
F#: Regular expressions/active patterns
Date: 2009-05-10 08:58:48

http://robotoverlords.blogspot.com/[Josh] has been http://twitter.com/markhneedham/status/1723697152[teaching me how to do regular expressions in Javascript this week] and intrigued as to how you would do this in F# I came across a couple of blog posts by Chris Smith talking about http://blogs.msdn.com/chrsmith/archive/2008/02/21/Introduction-to-F_2300_-Active-Patterns.aspx[active patterns] and http://blogs.msdn.com/chrsmith/archive/2008/02/22/regular-expressions-via-active-patterns.aspx[regular expressions via active patterns].

As I understand them active patterns are not that much different to normal functions but we can make use of them as part of a let or match statement which we can't do with a normal function.

I wanted to create an active pattern that would be able to tell me if a http://www.markhneedham.com/blog/2009/04/18/f-refactoring-that-little-twitter-application-into-objects/[Twitter] http://www.markhneedham.com/blog/2009/04/13/f-a-day-of-writing-a-little-twitter-application/[status] has a url in it and to return me that url. If there are no urls then it should tell me that as well.

This is therefore a partial active pattern as it does not necessarily describe something. Adapted from Chris Smith's blog I therefore ended up with the following active pattern:

[source,ocaml]
----

open System.Text.RegularExpressions

let (|Match|_|) pattern input =
    let m = Regex.Match(input, pattern) in
    if m.Success then Some (List.tl [ for g in m.Groups -> g.Value ]) else None
----

This is a generic active pattern which will take in a string and a regular expression and return an Option containing the matches if there are some and none if there aren't any.

The '_' in the active pattern definition is the partial bit - we don't necessarily have a match.

I quite liked what Chris did on line 4 of this statement whereby the results returned exclude the first item in the group of matches since this contains the entirety of the matched string rather than the individual matches.

I was then able to make use of the active pattern to check whether or not a Tweet contains a url:

[source,ocaml]
----

let ContainsUrl value =
    match value with
        | Match "(http:\/\/\S+)" result -> Some(result.Head)
        | _ -> None
----

Active patterns seem pretty cool from my limited playing around with them and are something that I came across by chance when looking around for ways to use regular expressions in F#.