markhneedham/blog huggingface dataset:
F#: Using C# extension methods
Date: 2009-06-15 20:03:34

An interesting thing I noticed about referencing C# libraries from F# is that you can't access C# extension methods on generic open types in the same way that you would be able to if you were using the library from C# code.

I came across this problem when playing around with the http://ayende.com/projects/rhino-mocks.aspx[Rhino Mocks framework] in some F# code.

I wrote a simple test to see whether I could get an expectation to work correctly, without paying any regard for the fact that you can't use all C# extension methods in the same way as you can from C# code!

[source,ocaml]
----

open Xunit
open Rhino.Mocks

type IFoo =
    abstract Bar : string -> string

type Foo() =
    interface IFoo with
        member x.Bar (value) = value

type Baz(foo: IFoo) =
    member x.Barry(value) = foo.Bar(value) |> ignore

[<Fact>]
let my_mocking_test () =
    let foo = MockRepository.GenerateMock<IFoo>()

	foo.Expect(fun f -> f.Bar("random")).Return("someValue")

    let baz = new Baz(foo)
    baz.Barry("random") |> ignore

    foo.VerifyAllExpectations();
----

That code doesn't compile with lines 18 and 23 being the offending ones.

If we want to use Rhino Mocks' extension methods in our F# code we need to call them specifically. With a little exploration through Rhino Mocks I came up with the following code.

[source,ocaml]
----

...
[<Fact>]
let my_mocking_test () =
    let foo = MockRepository.GenerateMock<IFoo>()
    (RhinoMocksExtensions.Expect<IFoo, string>(foo, fun f -> f.Bar("random"))).Return("someValue") |> ignore

    let baz = new Baz(foo)
    baz.Barry("random") |> ignore

    RhinoMocksExtensions.VerifyAllExpectations(foo)
----

It doesn't read particularly fluently although it does work. I imagine an equivalent extension method could be written in F# to get around the problem although I ended up not needing to do any mocking after I first wrote this code so I haven't looked into how to do that yet.

* Update *
I've updated this post to point out that this problem occurs when trying to use C# extension methods on open generic types rather than all C# extension methods, something which Matthew Podwysocki points out in the comments.

I had originally thought that you couldn't use any C# extension methods since they didn't show up on the Intellisense drop down in Visual Studio and I assumed they were just a language feature of C# 3.0.

As it turns out you can get the Intellisense by importing the 'System.Linq' namespace although if you want to use a C# extension method you will need to ensure the type has been evalualted which can sometimes mean you need to put brackets around the code.

For example:

[source,ocaml]
----

[1..5]. (* No extension methods will show up *)
([1..5]). (* All the extension methods that can be applied to IEnumerable show up  *)
----

Under the covers the same thing seems to be happening as if we called the extension methods from C# code. Scott Hanselman has a http://www.hanselman.com/blog/HowDoExtensionMethodsWorkAndWhyWasANewCLRNotRequired.aspx[nice post which explains how this works].
F#: Overlapping fields in record types
Date: 2009-06-14 00:37:01

A problem which has confused me for a while is how to create instances of record types whose fields overlap with another record defined further down in an F# file.

The most recently defined record seems to take precedence even if it has more fields than a record defined earlier and you don't specify all of those fields in your record creation attempt.

For example, if I have the following two record types:

[source,ocaml]
----

type Mark = { Value : string }
type SpecificMark = { Value: string; AnotherValue : int }
----

I thought it would be possible to create an instance of 'Mark' by doing this:

[source,ocaml]
----

let mark = { Value = "mark" }
----

But what I actually got is the following compilation error:

[source,text]
----

No assignment given for field 'AnotherValue'
----

It turns out that you need to be more explicit about the fact that you want to create the 'Mark' record type.

Using the pre F# 1.9.16 syntax we would use the following http://www.markhneedham.com/blog/2009/05/19/f-object-expressions/[object expression]:

[source,ocaml]
----

let mark = { new Mark with Value = "mark" }
----

But versions after that indicate that this syntax is deprecated and we need to do something more like this:

[source,ocaml]
----

let mark = { new Mark with member self.Value = "mark" }
----

Which leads to the following error:

[source,text]
----

error FS0191: Only simple bindings of the form 'id = expr' can be used in construction expressions
----

This has http://cs.hubfs.net/forums/thread/10412.aspx[also been mentioned on the hubfs forums] although there is no suggestion as for why this syntax would be invalid - it seems like it should work to me.

Luckily there are a couple of other ways to get an instance of the 'Mark' type:

[source,ocaml]
----

let (m:Mark) = { Value = "mark" }
----

[source,ocaml]
----

let mark = {Value = "mark" } : Mark
----

Both of which require less typing anyway!
Book Club: Working Effectively With Legacy Code - Chapter 11 (Michael Feathers)
Date: 2009-12-03 16:27:29

In our latest technical book club we discussed chapter 11 - 'I Need to Make a Change. What Methods Should I Test?' - of Michael Feathers' 'http://www.amazon.com/gp/product/0131177052?ie=UTF8&tag=marneesblo-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0131177052[Working Effectively With Legacy Code]'.

In this chapter Feathers covers some techniques which allow us to work out which parts of the code we need to write tests around when we make changes.

These are some of my thoughts and our discussion of the chapter:

* Feathers starts off the chapter by introducing the idea of *effect sketching* which I've been trying out recently both on the projects I've been working on and with the http://www.markhneedham.com/blog/2009/11/04/reading-code-unity/[Unity dependency injection container]. The idea is that we create a diagram which shows the effect that changing different fields and methods will have on the rest of the class and then based on this we can see where we need to create a safety net of tests for any given change. It seems like this type of diagram would also be quite useful for identifying when a class has more than one responsibility because you would end up with several mini unconnected effect sketches within the main one.
* Feathers suggests that we should look to write *characterisation tests* around legacy code. These are written a little differently to TDD's tests because we make the assertion match what the code is currently doing. The idea is to document what the system is doing now. We might come across 'bugs' while doing this but we wouldn't necessarily change the tests to verify how the code 'should' be until we find out what the effect of doing that would be. We want to write enough of these tests that we feel confident that we have a good enough safety net to allow us to change the code.
* There are some interesting ideas about encapsulation towards the end of the chapter where Feathers considers the *trade off between ensuring test coverage and encapsulation*.
+
____
Breaking encapsulation can make reasoning about our code harder, but it can make it easier if we end up with good explanatory tests afterward. \... Encapsulation and test coverage aren't always at odds, but when they are, I bias towards test coverage. Often it can help me get more encapsulation later. Encapsulation isn't an end in itself; it is a tool for understanding.
____
+
I find this is sometimes quite a big barrier to break down because people are often reluctant to change code to make it more testable. That's sometimes a valid concern but with legacy code not so much.
Coding: Naming
Date: 2009-12-16 22:08:22

Sarah Taraporewalla recently wrote http://sarahtaraporewalla.com/thoughts/design/the-importance-of-words[an interesting post about the importance of words with respect to the way that we use them in our code] and it reminded me of some conversations I've had with http://intwoplacesatonce.com/[Dave Cameron] about the importance of creating a shared understanding of the different types/objects in the systems that we build.

On a few projects that I've worked on where we didn't have a common understanding of what different concepts in the domain should be I noticed that there was a reluctance to make changes to class names.

Dave pointed out that this was probably because people have different maps in their head about what different domain concepts actually mean and changing class names ruins that map and therefore people's mechanism for remembering what different classes do.

== So how can we identify the real names?

If we can come up with a good enough way to name the types that exist in our system such that everyone has the same understanding then the mapping problem disappears.

Domain Driven Design suggests the http://devlicio.us/blogs/casey/archive/2009/02/09/ddd-the-ubiquitous-language.aspx[ubiquitous language] as a way to solve this problem, and while getting the language of the business into our code is the best way that I've come across for doing this sometimes it's difficult to get this sort of access to domain experts to make it possible.

In that situation I think just listening to how we talk about the code is enough to identify problem areas.

Typically if a concept isn't described by the name that it's been given then the person will describe the actual concept in the words they use to explain what they're actually referring to.

The neat thing about a ubiquitous language is that if we name things in such a way that everyone does have the same understanding then if we decide later that we need a new name by definition the new name will do a better name of describing an object than the old one.

Names are there to make it easier for us to talk and reason about code - if they're not doing that then we need to change them until they do.
F#: The use keyword and using function
Date: 2009-12-19 10:33:57

While I was playing around with http://www.markhneedham.com/blog/2009/12/18/f-word-count-a-somewhat-failed-attempt/[the little F# script that I wrote to try and solve the word count problem] I noticed that in a couple of places I had used the 'http://msdn.microsoft.com/en-us/library/dd233240(VS.100).aspx[use]' keyword when dealing with resources that needed to be released when they'd been used.

Using the 'use' keyword means that the 'Dispose' method will be called on the resource when it goes out of scope.

The two examples were 'StreamWriter' and 'StreamReader':

[source,ocaml]
----

let writeTo (path:string) f =
    use writer = new StreamWriter(path)
    f writer
----

[source,ocaml]
----

let download path =
    use streamReader = new StreamReader(File.OpenRead path)
    streamReader.ReadToEnd()
----

I found it quite annoying that those bits of code needed to take up three lines despite the fact I don't really need to have the class construction assigned.

Luckily there is a 'using' function available which allows us to make these bits of code more concise.

'using' takes in 2 arguments - the object to be created and a function which takes in that object and does something with it.

If we make use of that function instead of the 'use' keyword we end up with the following function definitions:

[source,ocaml]
----

let writeTo (path:string) f = using (new StreamWriter(path)) (fun writer -> f writer)
----

[source,ocaml]
----

let download path = using (new StreamReader(File.OpenRead path)) (fun reader -> reader.ReadToEnd())
----

When we're just doing one thing with the resource, as I am here, then I think this reads better. If we're using it for multiple different operations then perhaps the use keyword is more appropriate.
Haskell vs F#: Function composition
Date: 2009-12-09 22:10:27

I'm reading through John Hughes' 'http://www.cs.chalmers.se/~rjmh/Papers/whyfp.html[Why functional programming matters]' paper and one thing I've come across which is a bit counter intuitive to me is the Haskell function composition operator.

I've http://www.markhneedham.com/blog/2009/01/12/f-partial-function-application-with-the-function-composition-operator/[written previously about F#'s function composition operator] which is defined as follows:

[source,ocaml]
----

let inline (>>) f g x = g(f x)
----

To write a function which doubled all the values in a list and then returned the odd values we'd do this:

[source,ocaml]
----

let doubleThenOdd = List.map (fun x -> x*2) >> List.filter (fun x -> x % 2 <> 0)
----

Of course it's not possible for there to be any values!

[source,text]
----

doubleThenOdd [1..5];;
val it : int list = []
----

Based on that understanding I would expect the Haskell function composition operator ('.') to work in the same way:

[source,haskell]
----

let doubleThenOdd = map (\ x -> x*2) . filter (\ x -> (mod x 2) /= 0)
----

But it doesn't!

[source,text]
----

Prelude> doubleThenOdd [1..5]
[2,6,10]
----

In Haskell the functions are applied from right to left rather than left to right as I had expected.

The definition of '.' is therefore:

[source,text]
----

(f . g) x = f (g x)
----

So to get what I wanted we'd need to switch around 'map' and 'filter':

[source,haskell]
----

let doubleThenOdd = filter (\ x -> (mod x 2) /= 0) . map (\ x -> x*2)
----

[source,text]
----

Prelude> doubleThenOdd [1..5]
[]
----

It's not too difficult to follow once I worked out that it was different to what I was used to but I was very confused for a while!

Is there a reason why they implement this operator differently?
OOP: Behavioural and Structural constraints
Date: 2009-12-31 16:08:25

A few months ago I wrote a post describing how we should http://www.markhneedham.com/blog/2009/09/02/tdd-test-the-behaviour-rather-than-implementation/[test the behaviour of code rather than the implementation] whereby we would write tests against the public API of an object rather than exposing other internal data of the object and testing against that directly.

While I still think this is a useful way of testing code I didn't really have a good definition for what makes that a test of an object's behaviour.

I've been reading through James Odell's 'http://www.amazon.com/gp/product/052164819X?ie=UTF8&tag=marneesblo-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=052164819X[Advanced Object-Oriented Analysis and Design Using UML]' and he describes it like so:

____
Behavioural constraints limit the way object state changes may occur
____

In http://www.markhneedham.com/blog/2009/12/01/fundamentals-of-object-oriented-design-in-uml-book-review/[Meilir Page-Jones language] I think this would describe informative and imperative messages:

* Informative -- a message telling an object about something that happened in the past.
* Imperative -- a message telling an object to take some action on itself.

Both of these types of messages change the state of the object so in C# or Java these would be the public methods on an object that clients interact with.

That seems to describe the way that we would test the object. These would be the methods that we'd call in our test.

Odell goes on to describe structural constraints:

____
Structural constraints limit the way objects associate with each other, that is, they restrict object state.
____

This seems close to an interrogative message:

* Interrogative -- a message asking an object to reveal something about itself.

This would seem closer to the way that we would verify whether the object's state changed as expected. We're querying the object through the structural constraints that have been setup.

I can think of two main reasons why this approach is more effective than just testing directly against the internals of an object:

* It ensures we're testing something useful otherwise we might be writing tests on our code for a scenario that will never happen.
* We have a better idea of when we've finished writing our tests since we know when we've tested all the behaviour.
Clojure: Forgetting the brackets
Date: 2009-12-12 03:51:19

I've been playing around with macros over the last few days and while writing a simple one forgot to include the brackets to make it evaluate correctly:

[source,lisp]
----

(defmacro say-hello [person]
  println "Hello" person)
----

This macro doesn't even expand like I thought it would:

[source,text]
----

user=> (macroexpand-1 '(say-hello blah))
blah
----

That seemed a bit strange to me but I eventually realised that I'd missed off the brackets around 'println' and the arguments following it which would have resulted in 'println' being evaluated with those arguments.

I was a bit curious as to why that happened so I tried the following expression without any brackets to see what would happen:

[source,text]
----

user=> println "hello" "mark"
#<core$println__5440 clojure.core$println__5440@681ff4>
"mark"
"random"
----

It seems to just evaluate each thing individually and when we put this type of expression into a function definition the function will do the same thing but also return the last thing evaluated:

[source,lisp]
----

(defn say-hello [] println "hello" "mark")
----

[source,text]
----

user=> (say-hello)
"mark"
----

http://twitter.com/ajlopez/statuses/6563641565[A. J. Lopez pointed out] that this is quite like http://www.delorie.com/gnu/docs/elisp-manual-21/elisp_125.html[progn] in other LISPs and is the same as doing the following:

[source,text]
----

user=> (do println "hello" "mark")
"mark"
----

http://clojure.org/special_forms#toc3[do] is defined as follows:

____
(do exprs*) Evaluates the expressions in order and returns the value of the last. If no expressions are supplied, returns nil.
____

The way to write a function which passes those two arguments to 'println' is of course to put brackets around the statement:

[source,lisp]
----

(defn say-hello [] (println "hello" "mark"))
----

[source,text]
----

user=> (say-hello)
hello mark
nil
----
TDD: Only mock types you own
Date: 2009-12-13 21:47:04

http://lizdouglass.wordpress.com/2009/12/12/mock-objects/[Liz recently posted about mock objects] and the original 'http://www.jmock.org/oopsla2004.pdf[mock roles, not objects]' paper and one thing that stood out for me is the idea that *we should only mock types that we own*.

I think this is quite an important guideline to follow otherwise we can end up in a world of pain.

One area which seems particularly vulnerable to this type of thing is when it comes to http://www.markhneedham.com/blog/category/hibernate/[testing code which interacts with Hibernate].

A common pattern that I've noticed is to create a mock for the 'http://docs.jboss.org/hibernate/stable/entitymanager/reference/en/html_single/[EntityManager]' and then verify that certain methods on it were called when we persist or load an object for example.

There are a couple of reasons why doing this isn't a great idea:

. We have no idea what the correct method calls are in the first place so we're just guessing based on looking through the Hibernate code and selecting the methods that we think make it work correctly.
. If the library code gets changed then our tests break even though functionally the code might still work

The suggestion in the paper when confronted with this situation is to put a wrapper around the library and then presumably test that the correct methods were called on the wrapper.

____
Programmers should not write mocks for fixed types, such as those defined by the runtime or external libraries. Instead they should write thin wrappers to implement the application abstractions in terms of the underlying infrastructure. Those wrappers will have been defined as part of a need-driven test.
____

I've never actually used that approach but I've found that with Hibernate in particular it makes much more sense to write functional tests which *verify the expected behaviour of using the library*.

With other libraries which perhaps don't have side effects like Hibernate does those tests would be closer to unit tests but the goal is still to test the result that we get from using the library rather than being concerned with the way that the library achieves that result.
One change at a time
Date: 2009-12-22 06:01:04

I'm reading through Paul Butcher's 'http://www.amazon.com/gp/product/193435628X?ie=UTF8&tag=marneesblo-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=193435628X[Debug It]' book and one of his suggestions when trying to diagnose a problem in our code is to *only change one thing at a time*.

In a way this might seem fairly obvious but I've certainly fallen into the trap of making multiple changes at the same time in the misled belief that it'll lead to the problem being solved more quickly.

When making changes to code Butcher has the following piece of advice which I quite like:

____
Once you see a change in behavior, undo whatever apparently caused it, and verify that the behavior retur ns to what it was before-hand. This is a very power ful indication that you're looking at cause and effect rather than serendipity.
____

I noticed this while debugging my http://www.markhneedham.com/blog/2009/12/18/f-word-count-a-somewhat-failed-attempt/[F# word count application]. As I mentioned, http://www.markhneedham.com/blog/2009/12/20/f-word-count-using-a-dictionary/[I thought that the problem was that I was storing the text from all the files in memory] so instead of doing that I made that part of the application lazy so that the text would only be loaded when required.

When I first did this the program still didn't work but it failed later on than it had previously.

I thought that had shown where the problem was so I put the code back to how it was previously to check.

To my surprise it still failed in the same place which meant that the change in how it executed had been coincidental rather than related to any code change I'd made.

I think this idea is more widely applicable though as I've noticed that  http://www.markhneedham.com/blog/2009/02/08/refactoring-comment-it-out-vs-small-steps-removal/[it works quite well when refactoring] as well. If we can be really certain about which changes work and which don't to a very fine grained level then we have more chance of successfully refactoring our code while ensuring that it still functions correctly.

The goal seems to be the same in both of these situations - take small steps and then get feedback quickly on how successful that small step was.