markhneedham/blog huggingface dataset:
Google Maps without any labels/country names
Date: 2012-05-31 21:52:29

I wanted to get a blank version of Google Maps without any of the country names on for a visualisation I'm working on but I'd been led to believe that this wasn't actually possible.

In actual fact we do have control over http://stackoverflow.com/questions/3110020/google-maps-api-v3-no-labels[whether the labels are shown] https://developers.google.com/maps/documentation/javascript/styling#styling_the_default_map[via the 'styles' option] which we can call on the map.

In my case the code looks like this:

[source,javascript]
----

var map = new google.maps.Map(document.getElementById("map_canvas"), {
  zoom: 3,
  center: new google.maps.LatLng(31.492121, 14.919434),
  mapTypeId: google.maps.MapTypeId.ROADMAP
});

var emptyStyles = [
  {
    featureType: "all",
    elementType: "labels",
    stylers: [ { visibility: "off" } ]
  }
];

map.setOptions({styles: emptyStyles});
----

And this is the result:

image::{{<siteurl>}}/uploads/2012/05/map-no-labels.jpg[Map no labels,600]
Haskell: Java Style Enums
Date: 2012-05-30 11:10:15

I've been playing around with http://projecteuler.net/problem=31[problem 31 of Project Euler] which is defined as follows:

____
In England the currency is made up of pound, £, and pence, p, and there are eight coins in general circulation: 1p, 2p, 5p, 10p, 20p, 50p, £1 (100p) and £2 (200p). It is possible to make £2 in the following way: 1 £1 + 150p + 220p + 15p + 12p + 31p How many different ways can £2 be made using any number of coins?
____

Having coded way too much in Java my first thought was that the coins could be represented as an Enum but I wasn't sure how to do that in Haskell.

As it turns out http://stackoverflow.com/questions/6000511/better-way-to-define-an-enum-in-haskell[the question has been asked on Stack Overflow] and there's an Enum type class we can implement to achieve the same thing.

In this case:

[source,haskell]
----

data UKCoin = OnePence | TwoPence | FivePence | TenPence | TwentyPence | FiftyPence | OnePound | TwoPounds
              deriving (Eq, Show)
----

[source,haskell]
----

instance Enum UKCoin where
    fromEnum = fromJust . flip lookup table
    toEnum = fromJust . flip lookup (map swap table)
table = [(OnePence, 1), (TwoPence, 2), (FivePence, 5), (TenPence, 10), (TwentyPence, 20),
         (FiftyPence, 50), (OnePound, 100), (TwoPounds, 200)]
----

If we want to get the monetary value of a coin we use 'fromEnum':

[source,haskell]
----

> fromEnum TwoPounds
200
----

And if we wanted to go back to the enum value we'd do this:

[source,haskell]
----

> (toEnum 200) :: UKCoin
TwoPounds
----

If we want to get a list of all the instances of 'UKCoin' we can just do a map over 'table':

[source,haskell]
----

> let ukCoins = map fst table
> ukCoins
[OnePence,TwoPence,FivePence,TenPence,TwentyPence,FiftyPence,OnePound,TwoPounds]
----
Scala: Converting a scala collection to java.util.List
Date: 2012-02-05 21:40:33

I've been playing around a little with https://github.com/jiminoc/goose[Goose] - a library for extracting the main body of text from web pages - and I thought I'd try converting some of the code to be more scala-esque in style.

The API of the various classes/methods is designed so it's interoperable with Java code but in order to use functions like map/filter we need the collection to be a Scala one.

That's achieved by importing 'scala.collections.JavaConversions._' which will apply an implicit conversion to convert the Java collection into a Scala one.

I needed to go back to the Java one again which can be achieved with the following code:

[source,scala]
----

import scala.collection.JavaConversions._

val javaCollection = seqAsJavaList(Seq("abc"))
----

I also used that function in the https://github.com/mneedham/goose/blob/94da6fffc30db17a0a7e2060a46fc3317420a83e/src/main/scala/com/gravity/goose/text/StopWords.scala[StopWords.scala] object in Goose.

There are a load of other functions available in https://github.com/scala/scala/blob/master/src/library/scala/collection/JavaConversions.scala[JavaConversions] as well for going to a Dictionary, Map, Set and so on.
Tech Leads & The Progress Principle
Date: 2012-02-18 01:31:09

I've been reading http://www.amazon.co.uk/Progress-Principle-Ignite-Engagement-Creativity/dp/142219857X/ref=sr_1_1?ie=UTF8&qid=1329528627&sr=8-1[The Progress Principle] on and off for the last couple of months and one of my favourite quotes from the book is the following:

____
Truly effective video game designers know how to create a sense of progress for players within all stages of a game. Truly effective managers know how to do the same for their subordinates.
____

While a tech lead might not like to be referred to as a manager I think part of the role does involve *helping developers to make progress* and the best ones I've worked with seem to do that instinctively.

They're able to see when someone has got very stuck with what they're doing and can then work out whether they just need to provide some advice on how they can move forward or if that's not working they can come and work together on the problem.

____
Work progress and setbacks matter so much because work matters so much. It's simply part of being human
____

For the last couple of weeks a colleague and I have been trying to work out how to migrate reference data from an existing application for an application which we're rewriting.

We've run into problems along the way such as looking at a test database which we didn't realise was a couple of years out of date and therefore didn't have the data in the right format to trying to link up data from tables which weren't meant to be linked.

On each setback the tech lead (Alex) helped us progress first by recognising that the test and production databases might not be synced up and giving us other ideas about table design based on his greater knowledge of the whole system.

On this occasion although we were making some progress we couldn't quite figure out how to get the data into the shape we wanted which was becoming increasingly frustrating.

____
The effect of setbacks on emotions is stronger than the effect of progress\...the power of setbacks to diminish happiness is more than twice as strong as the power of progress to boost happiness.
____

Eventually Alex came and paired on the problem and was able to see something that we'd missed which allowed us to solve the problem.

I think the balance of initially allowing us to try and solve it ourselves and giving advice before taking a more hands on approach matches up fairly closely with what the authors of The Progress Principle suggest.

It's certainly been an interesting book to read and one which I think is very applicable to people working in software teams.
Java: Fooled by java.util.Arrays.asList
Date: 2012-02-11 10:29:15

I've been playing around with the http://code.google.com/p/boilerpipe/[boilerpipe] code base by writing some tests around it to check my understanding but ran into an interesting problem using +++<cite>+++java.util.Arrays.asList+++</cite>+++ to pass a list into one of the functions.

I was testing the +++<cite>+++https://github.com/mneedham/boilerpipe/blob/master/src/main/de/l3s/boilerpipe/filters/heuristics/BlockProximityFusion.java[BlockProximityFusion]+++</cite>+++ class which is used to merge together adjacent text blocks.

I started off calling that class like this:

[source,java]
----

import static java.util.Arrays.asList;

@Test
public void willCallBlockProximityFustion() throws Exception {
    TextDocument document = new TextDocument(asList(contentBlock("some words"), contentBlock("followed by more words")));
    BlockProximityFusion.MAX_DISTANCE_1.process(document);
}

private TextBlock contentBlock(String words) {
    TextBlock textBlock = new TextBlock(words, new BitSet(), wordCount(words), 0, 0, 0, 0);
    textBlock.setIsContent(true);
    return textBlock;
}
----

Which blows up like this:

[source,java]
----

java.lang.UnsupportedOperationException
	at java.util.AbstractList.remove(AbstractList.java:144)
	at java.util.AbstractList$Itr.remove(AbstractList.java:360)
	at de.l3s.boilerpipe.filters.heuristics.BlockProximityFusion.process(BlockProximityFusion.java:115)
	at de.l3s.boilerpipe.filters.heuristics.BlockProximityFusionTest.willCallBlockProximityFustion(BlockProximityFusionTest.java:63)
----

The code around that area is trying to remove an element from an iterator...

[source,java]
----

                if (ok) {
                    prevBlock.mergeNext(block);
                    it.remove();
                    changes = true;
                } else {
----

...which was created from the list that we passed into the constructor of +++<cite>+++TextDocument+++</cite>+++:
[source,java]
----

        for (Iterator<TextBlock> it = textBlocks.listIterator(offset); it
----

The +++<cite>+++remove+++</cite>+++ method is not implemented on the list created by 'Arrays.asList' which is weird since I thought it created an +++<cite>+++ArrayList+++</cite>+++ which does implement +++<cite>+++remove+++</cite>+++!

I've now learnt that the +++<cite>+++ArrayList+++</cite>+++ created by 'Arrays.asList' is actually a private inner class of +++<cite>+++Arrays+++</cite>+++ and doesn't implement the +++<cite>+++remove+++</cite>+++ method!

Who knew...
Downloading the JDK 6 source code
Date: 2012-02-11 10:02:09

Every now and then I want to get the JDK source code onto a new machine and it always seems to take me longer than I expect it to so this post is an attempt to help future me!

Googling for this takes me to http://download.java.net/jdk6/6u23/promoted/latest/[this page] and I always think I'll just checkout the http://java.net/projects/jdk-jrl-sources/[SVN repository] and hook that up but it doesn't seem to be available.

[source,text]
----

$ wget -S http://java.net/projects/jdk-jrl-sources/
--2012-02-11 09:51:34--  http://java.net/projects/jdk-jrl-sources/
Resolving java.net (java.net)... 192.9.164.103
Connecting to java.net (java.net)|192.9.164.103|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 404 Not Found
  Date: Sat, 11 Feb 2012 09:51:34 GMT
----

The alternative is therefore to download the jar provided which we can do like this:

[source,text]
----

wget http://www.java.net/download/jdk6/6u23/promoted/b05/jdk-6u23-fcs-src-b05-jrl-12_nov_2010.jar
----

The next step is then to execute the jar which I somehow didn't realise until I unpacked it and had a look at the README:

[source,text]
----

java -jar jdk-6u23-fcs-src-b05-jrl-12_nov_2010.jar
----

You get asked to choose a folder location for the sources and then the code is under 'src/share/classes'. So for me I need to give IntelliJ a source path of :

[source,text]
----

/Users/mneedham/github/j2se/src/share/classes
----

You can browse the different versions of the source code by changing the version number at the end of URLs like http://download.java.net/jdk6/6u23/[this one]. At the moment version 23 is the latest one available.
Haskell: Getting the nth element in a list
Date: 2012-02-28 00:02:21

I started trying to solve some of the http://projecteuler.net/problems[Project Euler problems] as a way to learn a bit of Haskell and http://projecteuler.net/problem=7[problem 7] is defined like so:

____
By listing the first six prime numbers: 2, 3, 5, 7, 11, and 13, we can see that the 6th prime is 13. What is the 10 001st prime number?
____

I read that the http://en.wikipedia.org/wiki/Sieve_of_Eratosthenes[Sieve of Eratosthenes] is a useful algorithm for working out all the prime numbers and there's http://en.literateprograms.org/Sieve_of_Eratosthenes_(Haskell)#chunk use:primes_naive[a page on the Literate Programs wiki explaining how to derive them using it].

The most naive implementation of all the primes ends up reading like this:

[source,haskell]
----

primes = 2 : sieve [3,5..]  where
    sieve []     = []
    sieve (p:xs) = p : sieve (xs `minus` [p,p+2*p..])
----

That gives an infinite sequence of all the prime numbers but I wanted to be able to specifically pick the 10,001st prime number which I assumed would be named 'nth' or something like that.

As it turns out we actually need to use the '!!' operator which I found out from http://scienceblogs.com/goodmath/2009/11/writing_basic_functions_in_has.php[Mark Chu-Carroll's blog post]:

[source,haskell]
----

*Main> :t (!!)
(!!) :: [a] -> Int -> a
----

[source,haskell]
----

problem_7 = primes !! 10000 -- 0 indexed so we need to get the position one before 10,001
----

It takes a while to run but we end up with the answer:

[source,haskell]
----

*Main> problem_7
104743
----
Java: Faking a closure with a factory to create a domain object
Date: 2012-02-26 00:09:03

Recently we wanted to create a domain object which needed to have an external dependency in order to do a calculation and we wanted to be able to stub out that dependency in our tests.

Originally we were just new'ing up the dependency inside the domain class but that makes it impossible to control it's value in a test.

Equally it didn't seem like we should be passing that dependency into the constructor of the domain object since it's not a piece of state which defines the object, just something that it uses.

We ended up with something similar to the following code where we have our domain object as an inner class:

[source,java]
----

public class FooFactory {
    private final RandomService randomService;

    public FooFactory(RandomService randomService) {
        this.randomService = randomService;
    }

    public Foo createFoo(String bar, int baz) {
        return new Foo(bar, baz);
    }

    class Foo {
        private String bar;
        private int baz;

        public Foo(String bar, int baz) {
            this.bar = bar;
            this.baz = baz;
        }

        public int awesomeStuff() {
            int random = randomService.random(bar, baz);
            return random * 3;
        }
    }
}
----

A test on that code could then read like this:

[source,java]
----

public class FooFactoryTest {
    @Test
    public void createsAFoo() {
        RandomService randomService = mock(RandomService.class);
        when(randomService.random("bar", 12)).thenReturn(13);

        FooFactory.Foo foo = new FooFactory(randomService).createFoo("bar", 12);
        assertThat(foo.awesomeStuff(), equalTo(39));
    }
}
----

It's a bit of a verbose way of getting around the problem but it seems to work reasonably well.
Optimising for typing
Date: 2012-02-21 22:21:43

My colleague Ola Bini recently wrote a post describing http://olabini.com/blog/2012/02/notes-on-syntax/[his thoughts on the syntax of programming languages] and while the post in general is interesting the bit that most resonates with me at the moment is the following:

____
Typing fewer characters doesn't actually optimize for writing either - the intuition behind that statement is quite easy: imagine you had to write a book. However, instead of writing it in English, you just wrote the gzipped version of the book directly. You would definitely have to type much less - but would that in any way help you write the book? No, probably it would make it harder. So *typing I definitely don't want to optimize*.
____

On the application I'm currently working on we have a full time DBA on the team who favours a much more concise style of naming in tables than most developers would be used to.

image::{{<siteurl>}}/uploads/2012/02/acronyms.gif[Acronyms,325]

We have an acronym for each table and if there's a foreign key reference to a field in that table elsewhere then we'll use the acronym as part of the column name in the other table.

For example if we had a reference to a table called +++<cite>+++Person+++</cite>+++ then our foreign key column name might be +++<cite>+++per_id+++</cite>+++.

One of the problems we were trying to overcome is that from what I understand https://forums.oracle.com/forums/thread.jspa?threadID=2141234[Oracle has a field name limit of 30 characters] so we would eventually exceed that if our naming was too verbose.

By using acronyms we can keep a more consistent style of naming rather than having some names being spelt out fully and others being abbreviated.

The other argument used for the more concise naming convention is that it's *much quicker to type* columns named in this way and we don't have auto complete functionality available in sqlplus as we do in most IDEs.

The disadvantage of the approach is that it's now very difficult to understand what data is being stored in a table just by looking at it - you need to first learn all the acronyms and then do the translations in your head.

If you're working with the tables all day long then it becomes second nature but if your use of them is more sporadic it takes a bit longer.

We're trying to keep interactions with the database reasonably minimal so hopefully we won't need to do any really complex queries with joins in which would be truly difficult to understand when we come back to them!
Delivery approach and constraints
Date: 2012-02-08 22:34:02

In my latest post I described an approach we'd been taking when analysing how to rewrite part of an existing system so that we could http://www.markhneedham.com/blog/2012/02/06/looking-for-the-seam/[build the new version in an incremental way].

Towards the end I pointed out that we weren't actually going to be using an incremental approach as we'd initially thought which was due to a couple of constraints that we have to work under.

== Hardware provisioning

One of the main reasons that we favoured an incremental approach is that we'd be able to deploy to production early which would allow us to show a quicker return on investment.

Unfortunately we later on came to learn that it takes *around 6-9 months to provision production hardware*.

It therefore didn't make a lot of sense to take an approach where we tried to integrate into the existing system since we wouldn't be able to deploy that work.

We're working under the assumption that in 6-9 months we'll probably be able to rewrite the whole thing and can therefore avoid the need to write the code which would allow us to integrate into the existing version.

We couldn't see any value in writing bridging code between the existing and new versions of the application if it never sees production - we'd put in all the effort for no reward.

== Running two systems side by side

Even if we had been able to provision hardware in time to release incrementally we came to learn that *the business would be uncomfortable with having two versions of the same application in production at the same time*.

The application is used to do pricing and the worry was that the different versions might produce different results for the same inputs.

It's arguably something we may have been able to overcome if we could prove that the new version worked exactly the same as the existing one by running both applications against a set of scenarios and checking that they returned the same results.

Theoretically I suppose the first problem could also be overcome but it's a battle we've chosen to leave alone for the moment.

What I found interesting in the discussions about the way we should deliver our solution was that I'd worked under the assumption that an incremental approach was always a better approach but with these constraints it isn't.
Looking for the seam
Date: 2012-02-06 22:22:16

During December/early January we spent some time analysing an existing system which we were looking to rewrite and our approach was to look for how we could do this in an incremental way.

In order to do that we needed to look for what Michael Feathers refers to as a http://www.markhneedham.com/blog/2009/06/21/seams-some-thoughts/[seam]:

____
A seam is a place where you can alter behaviour in your program without editing in that place
____

On previous times when I've been thinking about seams it's been at a code level inside a single application but this time there were more than one pieces interacting.

image::{{<siteurl>}}/uploads/2012/02/seam.gif[Seam,253]

We knew that there was a web application where the user could request a quote which would be calculated offline and then an email sent to them when it was ready to view.

That led us to believe that there was probably some sort of queue being used to store the outstanding requests and there'd probably be some sort of application processing the requests.

As it turned out the design of the system actually looked like the diagram on the right with the database effectively as a queue.

We then needed to work out which tables we had to read from/write to so that we'd be able to just replace the 'polling application' and leave the 'web application' alone.

We were then able to come up with a design whereby we isolated any interaction with the database into a 'bridging application' which then farmed requests out to a new application which we could scale horizontally.

It could also take care of writing the quotes back into the database so the existing application could read them back onto the screen.

Although we ended up not using this architecture for other reasons I think it's a neat way of looking at systems to work out how we can change them with minimal impact.
Thou shalt storm
Date: 2012-02-24 02:03:34

On the majority of the teams that I've worked on there's been a time where everyone seems to be disagreeing with each other about almost everything and the whole situation becomes pretty tense for all involved.

The first time I came across this it seemed quite dysfunctional but I was introduced to http://en.wikipedia.org/wiki/Tuckman's_stages_of_group_development[Bruce Tuckman's model of group development] which helps to explain what's going on.

Tuckman outlines four stages which teams tend to go through - *forming, storming, norming and performing*.

The forming stage tends to be fairly nice because people have mostly just met each other and don't want to cause any conflict but eventually we move into the storming stage which is where it gets interesting:

____
The team addresses issues such as what problems they are really supposed to solve, how they will function independently and together and what leadership model they will accept. Team members open up to each other and confront each other's ideas and perspectives. The storming stage is necessary to the growth of the team. It can be contentious, unpleasant and even painful to members of the team who are averse to conflict. Tolerance of each team member and their differences should be emphasized. Without tolerance and patience the team will fail.
____

In agile teams this conflict often tends to rear its head in a retrospective and people will tend to go away from it feeling pretty bad about the whole thing and wishing they could work on another team!

In reality it's just a stage that the team is going through and as long as team members address the differences of opinion they have rather than hiding from that conversation then the team can move on to being more productive.

One cool thing I've noticed is that if people have worked with each other before then they've probably already previously got over this hurdle and can therefore either skip/spend much less time on the storming stage.

A colleague of mine pointed out that we *shouldn't be alarmed* at the way people interact with each other in the storming stage but we *should be aware* when we're in it which I think is a pretty good way of looking at it.

Gina Abudi has http://www.pmhut.com/the-five-stages-of-project-team-development[an article from a couple of years ago] which goes through the storming stage and all the other ones in more detail.