markhneedham/blog huggingface dataset:
While in India: Osmotic communication
Date: 2011-01-24 03:33:39

One of the things that has been puzzling me during my time in India is the amount of time that is spent in meetings http://www.markhneedham.com/blog/2010/10/31/distributed-agile-context/[pushing information to people rather than them pulling it].

In previous projects that I've worked on a lot of the knowledge was moved between around as a result of http://alistair.cockburn.us/Osmotic+communication[osmotic communication]

____
Osmotic communication means that information flows into the background hearing of members of the team, so that they pick up relevant information as though by osmosis. This is normally accomplished by seating them in the same room. Then, when one person asks a question, others in the room can either tune in or tune out, contributing to the discussion or continuing with their work.
____

After thinking about it a bit more I've identified a few reasons why I think this approach doesn't work so well on my current project.

== The ability to ignore background noise

The ThoughtWorks Pune office is pretty much one big room with over 100 people in it so the amount of background noise is fairly constant.

When we've tried to run our standup near where we sit it's been impossible to hear people standing 5 or 6 metres away from you.

You therefore end up getting into the habit of blocking out anything going on around you as was evident when  http://www.markhneedham.com/blog/2010/12/15/india-cultural-differences-tolerancepatience/[there was drilling] being done in the office.

The side effect of doing this, of course, is that there could be relevant discussions going on around you and you'd be much less likely to notice.

== Team size/distribution

The second problem is that the team is distributed which means that we're never going to overhear any conversations in Chicago and vice versa.

To add to this our team is now 25+ in size so it's spread out over 3 tables one of which is only within shouting distance.

Martin Fowler wrote an article about http://martinfowler.com/bliki/TeamRoom.html[team rooms]  quite recently and while the ideas make sense I'm not sure how they scale as team size increases.

== Overlap of roles

Although this by no means applies to everyone that I've worked with I've noticed a tendency for people to stick to their role much more than I have in other countries.

Therefore a developer will only do 'developer tasks' and a QA will only do 'QA tasks' for example.

When you have overlap such that a developer may be 'pairing' with a QA or BA on something then the types of conversations that are head seem to lead to information moving around the team more effectively.
Installing git-svn on Mac OS X
Date: 2011-01-15 19:05:26

I somehow managed to uninstall git-svn on my machine and http://blog.emmanuelbernard.com/2009/01/how-to-install-git-and-git-svn-on-mac.html[Emmanuel Bernard's blog post] suggested it could be installed using ports:

[source,text]
----

sudo port install git-core +svn
----

I tried that and was ending up with the following error:

[source,text]
----

--->  Computing dependencies for git-core
--->  Dependencies to be installed: p5-svn-simple subversion-perlbindings apr-util db46 cyrus-sasl2 neon serf subversion p5-term-readkey
--->  Verifying checksum(s) for db46
Error: Checksum (md5) mismatch for patch.4.6.21.1
Error: Checksum (md5) mismatch for patch.4.6.21.2
Error: Checksum (md5) mismatch for patch.4.6.21.3
Error: Checksum (md5) mismatch for patch.4.6.21.4
Error: Target org.macports.checksum returned: Unable to verify file checksums
----

A bit of googling led me to the https://trac.macports.org/ticket/26075[macports website] where it was pointed out that this problem was happening in that user's case because the link to the patch now redirects to a HTML page:

____
It's actually the fact that the links pointing to those patches have become invalid. http://www.oracle.com/technology/products/berkeley-db/db/update/4.6.21/ redirects to the Berkley-DB webpage http://www.oracle.com/technetwork/database/berkeleydb/overview/index.html and you get a HTML instead of the real patch.
____

I'm not sure if that's the case for me because only the Checksum is failing, nothing is mentioned about downloading from Oracle's website.

By adapting one of the suggestions for downloading the patches manually I arrived at the following combination of commands to fix the db46 dependency:

[source,text]
----

cd /opt/local/var/macports/distfiles/db4/4.6.21_6/
sudo rm -rf patch*
for i in 1 2 3 4; do echo $i && sudo curl http://distfiles.macports.org/db4/4.6.21_6/patch.4.6.21.$i -O; done
----

Followed by:

[source,text]
----

sudo port install git-core +svn
----

And I'm back in the game!
mount_smbfs: mount error..File exists
Date: 2011-01-15 18:31:07

I've been playing around with mounting a Windows file share onto my machine via the terminal because I'm getting bored of constantly having to go to Finder and manually mounting it each time!

After a couple of times of mounting and unmounting the drive I ended up with this error:

[source,text]
----

> mount_smbfs //mneedham@punedc02/shared punedc02_shared/
mount_smbfs: mount error: /Volumes/punedc02_shared: File exists
----

I originally thought the 'file exists' part of the message was suggesting that I'd already mounted a share on 'punedc02_shared' but calling the 'umount' command led to the following error:

[source,text]
----

> umount punedc02_shared
umount: punedc02_shared: not currently mounted
----

I had actually absent mindedly gone and mounted the drive elsewhere through Finder which I only realised after reading http://www.cmsimike.com/blog/2009/06/30/os-x-failure-mounting-via-shell/comment-page-1/#comment-7714[Victor's comments on this post].

____
Make sure that you already do not have the same share mounted on your Mac. I had //host/share already mounted in /Volumes/share, so when I tried to mount //host/share to /Volumes/newshare it gave me the "`file exists`" error.
____

I learnt, thanks to the http://www.unix.com/71392-post2.html[unix.com forums], that you can see which devices are mounted by using 'df'.

This is where Finder had mounted the drive for me:

[source,text]
----

> df
Filesystem                 512-blocks      Used  Available Capacity  Mounted on
...
//mneedham@punedc02/shared  209696376 199773696    9922680    96%    /Volumes/shared
----

Since the shared drive gets unmounted when I disconnect from the network I decided to write a shell script that would set it up for me again.

[source,text]
----

#!/bin/sh
function mount_drive {
  mkdir -p $2
  mount_smbfs $1 $2
}

drives_to_unmount=`df | awk '/mneedham@punedc02/ { print $6 }'`

if [ "$drives_to_unmount" != "" ]; then
  echo "Unmounting existing drives on punedc02: \n$drives_to_unmount"
  umount $drives_to_unmount
fi

mount_drive //mneedham@punedc02/media /Volumes/punedc02_media
mount_drive //mneedham@punedc02/shared /Volumes/punedc02_shared
----

At the moment I've just put that in '/usr/bin' so that it's available on the path.

If there's a better way to do this or a way to simplify the code please let me know.

I did come across a few ways to do the mounting using Apple Script in http://hints.macworld.com/article.php?story=20020610225855377[this post] but that's not half as fun as using the shell!
Chris Argyris: Espoused Theory vs Theory in Action
Date: 2011-01-13 20:02:49

Via some combination of http://twitter.com/christianralph[Christian Blunden], http://twitter.com/#!/patkua[Pat Kua], http://twitter.com/#!/dpjoyce[David Joyce] and http://twitter.com/#!/benjaminm[Benjamin Mitchell] I've been spending some time lately reading about the work of http://blog.benjaminm.net/argyris/[Chris Argyris].

I've previously come across his name while reading http://www.amazon.com/Fifth-Discipline-Practice-Learning-Organization/dp/0385517254/ref=sr_1_1?ie=UTF8&qid=1294946576&sr=8-1[The Fifth Discipline] but I didn't realise how interesting his work actually is.

One of the interesting concepts I've come across so far is the difference between http://www.scu.edu.au/schools/gcm/ar/arp/argyris.html#a_arg_intro[espoused theory and theory in use]:

Espoused theory

____
The world view and values people believe their behaviour is based on.
____

Theory-in-use

____
The world view and values implied by their behaviour, or the maps they use to take action.
____

There are two areas that really stood out for me when I read these definitions.

== Interviews

In face to face interviews the candidate is likely to give an answer based on their espoused theory of the world and hence can come across as being very good if they know the type of answers you're looking for.

I've interviewed a couple of people over the last few years where I couldn't find fault with any of the answers being given but I was convinced that they weren't giving me an accurate picture of the candidate. The answers were too perfect.

Luckily we have an opportunity to get a closer look at a candidate's theory in action in the pair programming interview that we do.

I believe http://www.hashrocket.com/[HashRocket] take this even further by having candidates pair with their team for a week before they potentially get hired.

== Knowledge vs Experience

In a recent conversation with http://twitter.com/#!/davcamer[Dave Cameron] I was telling him about the difficulties I'd been having in applying the ideas I'd read in http://www.amazon.com/Crucial-Confrontations-Resolving-promises-expectations/dp/0071446524/ref=sr_1_1?ie=UTF8&qid=1294948454&sr=8-1[Crucial Confrontations], http://www.amazon.com/Agile-Coaching-Rachel-Davies/dp/1934356433/ref=sr_1_1?s=books&ie=UTF8&qid=1294948478&sr=1-1[Agile Coaching] and http://www.amazon.com/Fearless-Change-Patterns-Introducing-Ideas/dp/0201741571/ref=sr_1_1?s=books&ie=UTF8&qid=1294948497&sr=1-1[Fearless Change].

I understand the ideas that the books are suggesting but in a real life situation I nearly always make a mistake.

Dave pointed out that this is the difference between knowledge and experience - just because you know what to do doesn't mean that you will do it unless you've had some experience of the situation before.

This sounds  pretty similar to the difference between espoused theory and theory in action - I know what I want to do in a situation but at the moment that isn't what I actually do.
Sed: 'sed: 1: invalid command code R' on Mac OS X
Date: 2011-01-14 14:15:19

A few days ago I wrote about how we'd been http://www.markhneedham.com/blog/2011/01/11/sed-across-multiple-files/[using Sed to edit multiple files] and while those examples were derived from what we'd been using on Ubuntu I realised that they didn't actually work on Mac OS X.

For example, the following command:

[source,text]
----

sed -i 's/require/include/' Rakefile
----

Throws this error:

[source,text]
----

sed: 1: "Rakefile": invalid command code R
----

What I hadn't realised is that on the Mac version of sed the '-i' flag has a mandatory suffix, http://hintsforums.macworld.com/showpost.php?p=393450&postcount=11[as described in this post].

The appropriate section of the man page for sed on the Mac looks like this:

____
-i extension Edit files in-place, saving backups with the specified extension. If a zero-length extension is given, no backup will be saved. It is not recommended togive a zero-length extension when in-place editing files, as you risk corruption or partial content in situations where disk space is exhausted, etc.
____

Whereas on Ubuntu the suffix is optional so we see this:

____
-i[SUFFIX], --in-place[=SUFFIX] edit files in place (makes backup if extension supplied)
____

In order to get around this we need to provide a blank suffix when using the '-i' flag on the Mac:

[source,text]
----

sed -i "" 's/require/include/' Rakefile
----

I didn't http://en.wikipedia.org/wiki/RTFM[RTFM] closely enough the first time!
MarkLogic: Customising a result set
Date: 2011-06-20 22:36:48

One of the stories we worked on last week had us needing to be able to customise the output of a MarkLogic search query to include some elements which aren't included in the default view.

We started off with this:

search.xqy

[source,text]
----

xquery version "1.0-ml";
import module namespace search = "http://marklogic.com/appservices/search" at "/MarkLogic/appservices/search/search.xqy";

declare variable $term as xs:string := xdmp:get-request-field("query", "");

search:search($term)
----

Which gives us back a list of results showing where in the documents the search term appeared.

We wanted to be able to get the title of the document and some other meta data about it though so we needed to make use of the +++<cite>+++http://developer.marklogic.com/pubs/4.1/apidocs/SearchAPI.html[transform-results]+++</cite>+++ option to do this.

The original query changes to look like this:

.search.xqy
[source,text]
----
xquery version "1.0-ml";
import module namespace search = "http://marklogic.com/appservices/search" at "/MarkLogic/appservices/search/search.xqy";

declare variable $term as xs:string := xdmp:get-request-field("query", "");

search:search($term,
   <options xmlns="http://marklogic.com/appservices/search">
     <transform-results apply="transformed-result" ns="http://www.markhneedham.com/search" at="transform.xqy" />
   </options>)
----

And then we have another xquery file which we use to do the transformation:

.transfom.xqy
[source,text]
----

xquery version "1.0-ml";
module namespace custom-search = "http://markhneedham.com/search";
import module namespace search = "http://marklogic.com/appservices/search" at "/MarkLogic/appservices/search/search.xqy";

declare function custom-search:transformed-result(
         $result as node(),
         $ctsquery as schema-element(cts:query),
         $options as element(search:transform-results)?
) as element(search:snippet)
{
<search:snippet>
{
(
         search:snippet($result, $ctsquery, $options)/*,
         <extra>{$result//*:Id}{$result//*:Title}</extra>
)
}
</search:snippet>
};
----

We put the extra elements that we wanted to include in our search result in the +++<cite>++++++<extra>+++</cite> tag but we can name that anything we want.+++</extra>++++++</cite>+++

The code snippets assume that both +++<cite>+++search.xqy+++</cite>+++ and +++<cite>+++transform.xqy+++</cite>+++ are both at the top level of the documents database.
Chef, Fedora and 'ArgumentError: Attribute domain is not defined!'
Date: 2011-06-18 18:45:29

I've been playing around with http://wiki.opscode.com/display/chef/Chef+Solo[Chef Solo] on Fedora and executing the following:

[source,text]
----

sudo chef-solo -c config/solo.rb -j config/node.json
----

(where +++<cite>+++node.json+++</cite>+++ just contains the example code from the http://wiki.opscode.com/display/chef/Chef+Solo[resolver example on the Chef documentation page] and the cookbooks folder contains https://github.com/opscode/cookbooks/tree/master[all the opscode cookbooks].)

leads to the following error:

[source,text]
----

...
ERROR: Running exception handlers
ERROR: Exception handlers complete
FATAL: Stacktrace dumped to /home/mark/chef-solo/chef-stacktrace.out
FATAL: ArgumentError: Attribute domain is not defined!
----

A bit of googling led me to believe that this error is happening because http://lists.opscode.com/sympa/arc/chef/2010-03/msg00075.html[the machine doesn't have a fully qualified domain name (fqdn) defined] which can be seen by calling the following command:

[source,text]
----

$ hostname -f
> hostname: Name of service not known
----

One way to fix it is to add the following entry to +++<cite>+++/etc/hosts</etc>+++</cite>+++

[source,text]
----

127.0.0.1	mark-fedora
----

Which results in the script running fine with no errors.

[source,text]
----

...
INFO: Running report handlers
INFO: Report handlers complete
----

A suggestion I read while googling about fqdn was to add the hostname of the machine into a file called +++<cite>+++/etc/HOSTNAME+++</cite>+++ but that didn't seem to have any impact for me.

On the Mac +++<cite>+++hostname -f+++</cite>+++ works fine even without an entry like the above in +++<cite>+++/etc/hosts+++</cite>+++ so I'm not entirely sure how it all works!

If anyone could explain it to me that'd be awesome.
MarkLogic: Deleting all the documents in a database
Date: 2011-06-18 16:08:47

We're using the http://www.marklogic.com/[MarkLogic] database on my current project and something that we wanted to do recently was delete all the documents as part of a deployment script.

Getting all of the documents is reasonably easy - we just need to make a call to the +++<cite>+++doc()+++</cite>+++ function.

We can then iterate through the documents like so:

[source,text]
----

for $doc in doc() return $doc
----

We wanted to make use of the +++<cite>+++http://docs.marklogic.com/4.2doc/docapp.xqy#search.xqy?start=1&cat=all&query=xdmp:document-delete&button=search[xdmp:document-delete]+++</cite>+++ function to tear down all of the modules but that needs a uri representing the location of the document in the database which isn't available in +++<cite>+++$doc+++</cite>+++:

____
.xdmp:document-delete

*xdmp:document-delete*( $uri as xs:string ) as empty-sequence() Summary: Deletes a document from the database.
____

A colleague pointed out that what we needed to do was pass the document to +++<cite>+++http://docs.marklogic.com/4.2doc/docapp.xqy#search.xqy?start=1&cat=all&query=xdmp:node-uri&button=search[xdmp:node-uri]+++</cite>+++ and then our troubles would be over!

The final solution therefore looks like this:

[source,text]
----

for $doc in doc() return xdmp:document-delete(xdmp:node-uri($doc))
----

I was expecting that we would delete the documents by some sort of identifier but I guess this approach makes more sense given the way the data is stored.

It all seems a bit esoteric at the moment!
Scala: Self type annotations and structured types
Date: 2011-06-27 23:21:56

A few days ago I http://twitter.com/markhneedham/status/84025938903437312[tweeted that I didn't really see the point in structured types in Scala]...

____
Not sure I understand where you would use structural types in #scala instead of defining a method on a trait http://bit.ly/jgiW7b
____

...but today my colleague Uday came up with a cool way of combining http://programming-scala.labs.oreilly.com/ch12.html#SelfTypeAnnotations[self type annotations] with structured types inside a trait we defined.

We had some code duplicated across two classes which looked roughly like this:

[source,scala]
----

class OnePageType {
  lazy val peopleNodes = root \\ "SomeNode" \ "SomeSubNode" \ "People" \ "Person"
  private def fullName(personName: Node): String = // code to build person's name

  lazy val people: String = peopleNodes.map(fullName).mkString(", ")
}
----

[source,scala]
----

class AnotherPageType {
  lazy val peopleNodes = root \\ "OtherNode" \ "OtherSubNode" \ "People" \ "Person"
  private def fullName(personName: Node): String = // code to build person's name

  lazy val people: String = peopleNodes.map(fullName).mkString(", ")
}
----

The first line is different but the other two are identical because the data is stored in exactly the same format once we get down to that level.

Since We want to keep the XPathish queries as descriptive as possible so that we don't accidentally end up pulling the wrong elements onto the page, making those a bit looser wasn't an option in this case.

Instead we pulled out a trait like so:

[source,scala]
----

trait People {
  self: {val peopleNodes: NodeSeq} =>

  private def fullName(personName: Node): String = // code to build person's name

  lazy val people: String = peopleNodes.map(fullName).mkString(", ")
}
----

Which we include in the classes like this:

[source,scala]
----

class OnePageType extends People {}
class AnotherPageType extends People {}
----

What we're done on line 2 of the +++<cite>+++People+++</cite>+++ trait is to define a self annotation which says that we need a val of +++<cite>+++peopleNodes+++</cite>+++ to be present on the classes in which the trait is mixed.

If a val of +++<cite>+++peopleNodes+++</cite>+++ doesn't exist then the class won't compile!

In this case the structure type works quite well because we wouldn't really want to pull out +++<cite>+++peopleNodes+++</cite>+++ into a trait just to reference it as a self type annotation.
Fedora: Recovering from the IntelliJ 'Ctrl-Alt-F7'
Date: 2011-06-16 07:27:15

We're using Fedora on our local developer work stations and some of the default key bindings of the operating system seem to conflict with ones provided by IntelliJ IDEA.

One particular amusing one is 'Ctrl-Alt-F7' which you use in IntelliJ to see the usages of a piece of code.

In Fedora that seems to switch into a different X Server session and you just see a blank screen with seemingly no way out!

Pressing 'Ctrl-Alt-F1' helps us get back to our original session again.

'''

I'm no expert of all things Unix so if I've described anything incorrectly feel free to point it out in the comments.
Git: Deleting a remote branch on a gitolite configured repository
Date: 2011-06-28 22:09:18

We've had an https://github.com/harrah/xsbt[xsbt] branch on our https://github.com/sitaramc/gitolite[gitolite] powered repository for the last couple of weeks while we worked out how to move our build from sbt 0.7 to sbt 0.10 but having finally done that we needed to delete it.

I originally tried running the following command from one of our developer workstations:

[source,text]
----

git push origin :xsbt
----

But ended up with the following error:

____
remote: error: denying ref deletion for regs/head/xsbt ! [remote rejected] xsbt (deletion prohibited)
____

A bit of googling led me to http://stackoverflow.com/questions/5723511/how-do-i-remove-a-remote-branch-when-i-get-an-error[this stackoverflow thread] which suggested that you needed to be an administrator in order to delete a remote branch.

Once we've done that we can run the following command on each machine to delete the remote tracking reference to the repository:

[source,text]
----

git branch -d -r origin/xsbt
----
Sbt: Rolling with continuous/incremental compilation and Jetty
Date: 2011-06-10 00:16:05

As I mentioned http://www.markhneedham.com/blog/2011/06/04/sbt-zipping-files-without-their-directory-structure/[in an earlier post] we're using SBT on our project and one of it's cool features is that it will listen to the source directory and then automatically recompile the code when it detects file changes.

We've also installed the https://github.com/glenford/sbt-jetty-embed[sbt-jetty-embed] plugin which allows us to create a war which has Jetty embedded so that we can http://patforna.blogspot.com/2011/04/containerless.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+patforna+%28Patric+Fornasier%27s+Blog%29[keep our application containerless].

That plugin adds an action called 'jetty' to sbt so we (foolishly in hindsight) thought that we would be able to launch the application in triggered execution mode by making use of a ~ in front of that:

[source,text]
----

$ ./sbt
> ~jetty
----

Unfortunately that doesn't do any continuous compilation which left us quite confused until we realised that http://code.google.com/p/simple-build-tool/wiki/TriggeredExecution[RTFM] might be a good idea...

What we actually needed to launch the application locally and edit code/see changes as if it's written in a dynamic language was the following:

[source,text]
----

$ ./sbt
> jetty-run
> ~ prepare-webapp
----