markhneedham/blog huggingface dataset:
Scala: Do modifiers on functions really matter?
Date: 2011-08-13 02:10:53

A couple of colleagues and I were having an interesting discussion this afternoon about the visibility of functions which are mixed into an object from a trait.

The trait in question looks like this:

[source,scala]
----

trait Formatting {
  def formatBytes(bytes: Long): Long = {
    math.round(bytes.toDouble / 1024)
  }
}
----

And is mixed into various objects which need to display the size of a file in kB like this:

[source,scala]
----

class SomeObject extends Formatting {

}
----

By mixing that function into +++<cite>+++SomeObject+++</cite>+++ any of the clients of +++<cite>+++SomeObject+++</cite>+++ would now to be able to call that function and transform a bytes value of their own!

The public API of +++<cite>+++SomeObject+++</cite>+++ is now cluttered with this extra method although it can't actually do any damage to the state of +++<cite>+++SomeObject+++</cite>+++ because it's a pure function whose output depends only on the input given to it.

There are a couple of ways I can think of to solve the modifier 'problem':

* Make +++<cite>+++formatBytes+++</cite>+++ a private method on +++<cite>+++SomeObject+++</cite>+++
* Put +++<cite>+++formatBytes+++</cite>+++ on a singleton object and call it from +++<cite>+++SomeObject+++</cite>+++

The problem with the first approach is that it means we have to test the +++<cite>+++formatBytes+++</cite>+++ function within the context of +++<cite>+++SomeObject+++</cite>+++ which makes our test much more difficult than if we can test it on its own.

It also makes the discoverability of that function more difficult for someone else who has the same problem to solve elsewhere.

With the second approach we'll have a dependency on that singleton object in our object which we wouldn't be able to replace in a test context even if we wanted to.

While thinking about this afterwards I realised that it was quite similar to something that I used to notice when i was learning F# - the modifiers on functions don't seem to matter if the data they operate on is immutable.

I often used to go back over bits of code I'd written and make all the helper functions private before realising that it made more sense to keep them public but group them with similar functions in a module.

I'm moving towards the opinion that if the data is immutable then it doesn't actually matter that much who it's accessible to because they can't change the original version of that data.

+++<cite>+++private+++</cite>+++ only seems to make sense if it's a function mutating a specific bit of data in an object but I'd be interesting in hearing where else my opinion doesn't make sense.
MySQL: The used command is not allowed with this MySQL version
Date: 2011-01-18 18:58:35

For my own reference more than anything else, on my version of MySQL on Mac OS X, which is:

____
mysql5 Ver 14.14 Distrib 5.1.48, for apple-darwin10.4.0 (i386) using readline 6.1
____

When I try to use the 'LOAD DATA LOCAL' option to load data into tables I get the following error message:

[source,text]
----

ERROR 1148 (42000) at line 4: The used command is not allowed with this MySQL version
----

Which we can get around by using the following flag as described in the http://dev.mysql.com/doc/refman/5.0/en/loading-tables.html[comments of the documentation]:

[source,text]
----

mysql --local-infile -uroot -pandsoon
----

Or by putting the following entry in a config file (in my case ~/.my.cnf):

[source,text]
----

[mysql]
local-infile
----

I tried a bit of googling to see if I could work out why it happens but I'm still none the wiser.
Increasing team sizes: Boredom
Date: 2011-01-27 22:59:37

Although the majority of the teams that I've worked on over the past few years have been relatively small in size I have worked on a few where the team size has been pretty big and perhaps inevitably the productivity has felt much lower.

I think this is somewhat inevitable since although the overall throughput of these teams may be higher than on smaller teams, due to problems such as http://www.markhneedham.com/blog/2010/11/26/increasing-team-sizes-parallelising-work/[having difficulty parallelising work], not every pair is working at maximum productivity.

This tends to mean that you'll end up with some pairs who have development work to do for only part of the iteration and will then be unable to pick up a new story because there are dependencies on other stories which are currently in progress.

As a result the people in this position will get extremely bored and eventually they'll start to distract the other people on the team which means that their productivity goes down as well.

Another side effect of knowing that there isn't anything else for you to pick up is that you subconsciously won't finish the story you're working on as quickly as you otherwise might be able to.

One suggestion I've heard is that people should pick up technical debt when they find themselves in this situation but from what I've noticed unless you were the one who noticed the technical debt there is both context and motivation lacking.

Another possibility in this type of situation is for the developers to try and help out elsewhere perhaps by doing some *technical analysis on upcoming stories* or *helping out testers with their work*.

The former seems to work reasonably well but the problem when doing this offshore is that you can often only get up to a certain point at which you need some help from guys onshore.

As a result technical analysis often isn't enough to keep a pair occupied for that long.

I've previously seen the latter work reasonably well whereby a tester would help the developer to work out which unhappy path scenarios needed to be automated and the developer could then write the automation code.

I haven't seen that work particularly well on my current team and I think the reason is probably down to the fact that we keep separate sets of 'dev' and 'QA' functional tests.

I can't currently think of a good solution to this problem other than matching the amount of work and the number of pairs more carefully so that everyone is occupied for the majority of the time.
Rails: Using helpers inside a controller
Date: 2011-01-11 17:09:49

For about an hour or so this afternoon we were following the somewhat evil practice of using a method defined in a helper inside a controller.

The method was defined in the ApplicationHelper module:

[source,ruby]
----

module ApplicationHelper
	def foo
		# do something
	end
end
----

So we initially assumed that we'd just be able to reference that method inside any of our controllers since they all derive from ApplicationController.

That wasn't the case so our next attempt was to try and add it as a helper:

[source,ruby]
----

class FooController < ApplicationController
	helper :application
end
----

Which makes it accessible from the view but not from the controller...

Eventually we called link:[Ashwin] to help us out and he came across http://snippets.dzone.com/posts/show/1799[this thread on dzone].

About half way down the page ovhaag points out that we can use '@template' to get access to helper methods:

____
In any controller, there is a "@template"-instance and you can call helper methods on this. I found this trick in http://media.railscasts.com/videos/132_helpers_outside_views.mov \... Ryan is not sure if this use is intended but it is very short and today it works.
____

We can use that instance variable like so:

[source,ruby]
----

class FooController < ApplicationController
	def our_method
		# We can call foo like this
		@template.foo
	end
end
----

We eventually found out another way to do what we wanted but it seems like a neat little trick
Sed across multiple files
Date: 2011-01-11 16:43:53

http://pankhurisa.blogspot.com/[Pankhuri] and I needed to rename a method and change all the places where it was used and decided to see if we could work out how to do it using sed.

We needed to change a method call roughly like this:

[source,ruby]
----

home_link(current_user)
----

To instead read:

[source,ruby]
----

homepage_path
----

For which we need the following sed expression:

[source,text]
----

sed -i 's/home_link([^)]*)/homepage_path/' [file_name]
----

Which works pretty well if you know which file you want to change but we wanted to run it over the whole code base.

A bit of googling led us to http://forums.devshed.com/unix-help-35/how-to-use-sed-to-search-replace-files-throughout-a-184662.html[this thread on devshed] which suggested we'd need to get a list of the files and then run sed through the list:

[source,text]
----

for file in `find .  -type f`; do sed -i 's/home_link([^)]*)/homepage_path/' $file; done
----

That pretty much works but it doesn't play nicely if the file has a space in the name since sed thinks the file name has ended before it actually has.

I was pretty sure that we should be able to pipe the output of the find into xargs and a bit more googling led us to the following solution:

[source,text]
----

find . -type f -print0 | xargs -0 sed -i 's/home_link([^)]*)/homepage_path/'
----

The 'print0' flag is described like so:

[source,text]
----

This primary always evaluates to true.  It prints the pathname of the current file to standard output, followed by an ASCII NUL character (character code 0).
----

While '-0' in 'xargs' is described like this:

[source,text]
----

  -0      Change xargs to expect NUL (``\0'') characters as separators, instead of spaces and newlines.  This is expected to be used in concert with the -print0 function in find(1).
----

It also runs amazingly fast!

If anyone knows a better way feel free to point it out in the comments.
Jet Airways: Lacking conceptual integrity and the power of twitter
Date: 2011-01-10 18:08:22

I recently travelled to London and back for Christmas using Jet Airways and the whole journey got off to an 'interesting' start.

I originally booked two Jet Airways flights - one from Pune to Delhi and another from Delhi to London.

A couple of weeks later I was sent an email cancelling my Pune to Delhi flight and informing me that I should contact their customer support centre.

I quickly browsed their website to check what had happened to my flight and found out that it had actually changed from being a Jet Airways flight and was now in fact a Jet Lite flight - their sister airline.

I called the customer support centre and asked them to book me that flight which was fine and I was told that I'd be sent an email with the updated itinerary.

I didn't receive the email which normally wouldn't be a big deal except you can't get into Indian airports unless you have a print out containing the details of the flight that you're going to be taking.

I called the customer support centre again and was told that I'd need to come to the airport and go to the Jet Airways information desk in order to get a print out of my new itinerary.

I went there and was given a copy of what I thought was my new itinerary. I didn't realise until later on that I'd actually been given a copy of an itinerary still containing my old flight which had been cancelled.

Presumably the system which the Jet Airways staff were using doesn't contain information about Jet Lite which to me as a customer shows+++<strong>+++a lack of conceptual integrity+++</strong>+++ because to me they're just one organisation even though internally they may not be.

I decided to see if I could sort it out on twitter and after a few messages back and forth with whoever controls the http://twitter.com/#!/jetairways[Jet Airways twitter account] I got an email with my new itinerary.

Hurray for twitter!
Failure of integration point doesn't have to stop the user: A real life example
Date: 2011-01-10 15:28:44

link:[Ashwin] and I were recently discussing integration points in software systems and in particular how many systems are designed in such a way that they will stop the user from going any further if one of those integration points is down.

The main point in favour of designing systems in this way is that it's logically very simple - all operations are synchronous and we don't have to worry about any offline processing.

That doesn't mean that it gives the best user experience though and the more integration points that we have the more likely the user will experience a problem at some stage.

I vaguely remember hearing/reading someone (I think by http://www.michaelnygard.com/[Michael Nygaard] but if anyone knows please let me know!) point out that we often don't need all of the integration points to be up in order to continue the user's journey.

Of course there might be some user interactions with integration points where we do need that system to be online but for others, which play a more supporting role, we can just keep going and then process anything related to that system offline later on.

'http://www.damninteresting.com/the-baader-meinhof-phenomenon[Coincidentally]' later that day I was using an RBS ATM which informed me that the receipt printer wasn't working and asked me if I'd like to continue with my transaction.

The receipt in a cash withdrawal transaction is effectively a supporting 'feature' so it's reasonable that I should be able to continue with what I was doing.

Obviously if the machine was out of money then it wouldn't make sense to allow me to continue with a cash withdrawal transaction but it would still be fine for me to do a balance enquiry for example.

I haven't been at the ATM when it was out of money to check whether it actually does!
Deliberate Discovery: The stuff I don't know list
Date: 2011-01-26 18:07:07

Towards the end of Dan North's post on http://dannorth.net/2010/08/30/introducing-deliberate-discovery/[Deliberate Discovery] he makes the following suggestion:

____
There is much more to say about deliberate discovery. Think about applying the principle to learning a new language, or picking up a new technology, or a new domain. What could you do to identify and reduce your ignorance most rapidly?
____

This reminded me a lot of what I used to do when I came across things that I didn't know how to do a few years ago.

I would write down the thing that I didn't know and then make sure that I looked up the things on my list at the end of the day and then discuss what I'd read/learnt with a colleague the next day.

More recently I've fallen into what I'd call a complacent pattern where if I find something that I don't know 100% how to do I don't go out of my way to check that I actually know how to do it now.

I have more confidence that I'll be able to work out how to do something if the need arises but I don't gain that knowledge right away.

Effectively *I've continued identifying my ignorance but haven't taken the steps to reduce it*.

Now that I've realised I'm doing this I've started writing down things that I don't know on a "Stuff I don't know list" and I'm back to spending some time each evening to run through the list and at least gain some knowledge on each item.

One problem I used to find when I did this before was that I'd start reading about one thing and then I'd notice something else that I didn't know, go off and read about that and so on until I'd forgotten what I was originally trying to learn.

To try and combat that I've been time-boxing myself with a http://www.pomodorotechnique.com/[pomodoro] while I'm working through the list and then adding any new concepts I come across to the end of the list.

Another mistake I've made/sometimes do make is reading very passively i.e. I read the information but don't really understand it.

To get around that problem I've been talking to colleagues about what I've been reading, as suggested by Andy Hunt in http://www.markhneedham.com/blog/2008/10/06/pragmatic-learning-and-thinking-book-review/[Pragmatic Thinking and Learning].

In addition I tend to remember things better if I actually get to use them in practice rather than just learning the theory.

Since the majority of the reading I've been doing lately has been around networking I've been using http://openmaniak.com/tcpdump.php[tcpdump] and http://www.wireshark.org/[wireshark] so I can see the theory in action.

Unfortunately those are another two things I don't know so I'm learning about them as I go.
Listening to feedback mechanisms
Date: 2011-01-21 03:46:21

In http://static.mockobjects.com/labels/listening%20to%20the%20tests.html[Growing Object Oriented Software] the authors talk about the value of listening to our tests to understand potential problems with our code and I've started to notice recently that there are implicit feedback mechanisms dotted around at a higher level which we can also listen to.

A couple of examples come to mind:

== Nothing to show in the showcase

I've worked on a couple of projects where we've got to the end of the iteration and realised that we don't actually have anything tangible to show the product owner.

This tends to be because we've spent the majority of our time working on 'technical stories' which typically have no visible output.

This may be as a result of horizontal slicing of work rather than vertical slicing but what it essentially means is that the work you've done has no immediate value to anyone.

Sometimes there are technical aspects to a solution which have a lot of unknowns and combining those with a vertically sliced story would mean that the story becomes too big to fit in an iteration.

From my experience a way around this which has worked well before is to run a spike/experiment card where we work out all the technical details/gotchas and setup any necessary 'framework' and then have the other vertically sliced chunks of work plug into this.

Having said that I'm no expert in breaking down work into manageable chunks so if anyone has any other ideas I'd be interested in hearing about those.

== Boring meetings

The typical response if someone complains of http://boycottameetingday.com/[being bored in a meeting] or walks out of a meeting is that they should deal with it but that response doesn't help us that much.

I think if people are bored then it's an indication that the meeting isn't particularly interactive/inclusive or that the person shouldn't be in the meeting.

We can try to resolve that first point by having meetings which are designed more as a discussion/learning forum rather than a brain dump session by one person to the group.

This encourages more of a http://www.markhneedham.com/blog/2010/10/31/distributed-agile-context/[pull approach over a push] one and seems to be more effective at keeping people's attention as long as the topic of the discussion is actually relevant to them.

Frequently it seems that a more effective approach would be to have fewer people involved in a meeting and have one of the participants quickly summarise what happened for the rest of the team.
Ruby: Sorting by boolean fields
Date: 2011-01-08 13:15:19

We were doing a bit of work on http://rapidftr.com/[RapidFTR] in the ThoughtWorks Pune office today and one problem my pair and I were trying to solve was how to sort a collection of objects by a boolean field.

Therefore given the following array of values:

[source,ruby]
----

form_sections = [FormSection.new(:enabled => false, :name => "a", :order => 1),
                 FormSection.new(:enabled => true, :name => "b", :order => 2)]
----

We wanted to display those form sections which were disabled at the bottom of the page.

We originally tried the following:

[source,ruby]
----

form_sections.sort_by { |row| [row.enabled, row.order] }
----

But got the following error:

[source,ruby]
----

undefined method `<=>' for true:TrueClass
----

We figured we'd need to convert the true and false values to equivalent values which is demonstrated on http://stackoverflow.com/questions/903877/sort-objects-by-boolean-values-in-ruby[this Stack Overflow post]:

[source,ruby]
----

form_sections.sort_by { |row| [row.enabled ? 0 : 1, row.order] }
----

I didn't realise it would be that simple to do - it's pretty neat.
Distributed Agile: Stories -  Negotiable
Date: 2011-01-24 03:34:47

I was recently reading http://www.subcide.com/articles/how-to-write-meaningful-user-stories/[an article about how to write meaningful user stories] and towards the end of it the author mentioned the INVEST acronym which suggests that stories should be:

. Independent
. Negotiable
. Valuable
. Estimable
. Small
. Testable

From what I've seen the most difficult one to achieve in a distributed context is that stories should be 'negotiable', in particular when it comes to negotiating the way that the UX of a bit of functionality should work.

On most of the projects that I've worked on the people designing the UX tend to work slightly detached from the development team and then send their designs over as wire frames.

It's not the most ideal setup even if you're working onshore but it becomes even more challenging when you're working offshore.

Typically onshore if a particular user flow was very difficult to implement then one of the developers might go and talk with the UX guys and then explain the problem and give another potential solution.

The trade off between the cost of implementation and the user experience that the UX person has suggested is clearly outlined in these conversations.

When that feedback comes from offshore it has much less impact and I think it comes across much more as a criticism of someone's work rather than an attempt to be pragmatic in helping the client to deliver a product within a time frame.

One possible solution to this problem if you have some onshore colleagues is to have them go and talk about the problem but we've found it difficult to do that because we try to split onshore/offshore stories so that we're working on different parts of the code base.

Therefore it is pretty difficult for an onshore developer to go and discuss it for you.

To add to the problem we tend to realise the difficult of implementing a certain UX during the middle of our day which means we need to wait half a day to see if we can get it changed.

Given that time lag we often just end up designing it the way that's been specified so that we don't 'waste' time.

It's clearly not an idea situation so I'd be keen to hear if anyone has come up with ideas to get around this.