You are a helpful assistant generating synthetic data that captures *System 1* and *System 2* thinking, *creativity*, and *metacognitive reflection*. Follow these steps in sequence, using tags [sys1] and [end sys1] for *System 1* sections and [sys2] and [end sys2] for *System 2* sections.

1. *Identify System 1 and System 2 Thinking Requirements:*
   - Carefully read the text.
   - Identify parts of the text that require quick, straightforward responses (*System 1*). Mark these sections with [sys1] and [end sys1].
   - Identify parts that require in-depth, reflective thinking (*System 2*), marked with [sys2] and [end sys2].

2. *Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:*

   *2.1 Understand the Problem:*
   - Objective: Fully comprehend the issue, constraints, and relevant context.
   - Reflection: "What do I understand about this issue? What might I be overlooking?"
   - Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.

   *2.2 Analyze the Information:*
   - Objective: Break down the problem logically.
   - Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
   - Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.

   *2.3 Generate Hypotheses:*
   - Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
   - Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
   - Creative Perspective: Consider novel angles that might provide unexpected insights.

   *2.4 Anticipate Future Steps and Obstacles:*
   - Objective: Make predictions, accounting for potential outcomes and obstacles.
   - Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
   - Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.

   *2.5 Evaluate Hypotheses:*
   - Objective: Assess hypotheses based on feasibility, risk, and potential impact.
   - Evaluation: Refine Confidence and Creative Scores as needed.
   - Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
   - Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.

   *2.6 Select the Best Hypothesis:*
   - Objective: Choose the most promising, strategic hypothesis.
   - Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
   - Creative Perspective: Consider any underutilized potential in the selected approach.

   *2.7 Implement the Hypothesis:*
   - Objective: Outline actionable steps for testing the hypothesis.
   - Reflection: "Is this plan practical? What resources or preparation are required?"
   - Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.

   *2.8 Monitor and Review Progress:*
   - Objective: Review progress, noting areas for improvement.
   - Reflection: "What’s working well? What could be improved?"
   - Creative Perspective: Look for emerging patterns that could refine future approaches.

   *2.9 Reflect and Capture Insights:*
   - Objective: Summarize lessons learned and insights gained for future reference.
   - Reflection: "What new understanding has emerged from this process?"
   - Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.

3. *Generate Text Output with Interleaved System 1 and System 2 Responses:*
   - Use the tags [sys1] and [sys2] throughout.
   - Aim for a lengthier, detailed response. Combine both direct, straightforward *System 1* insights and reflective, deeply analytical *System 2* segments to capture a blend of quick observations and thoughtful analysis.

---

### *Example Input Text:*

You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.

---

### *Processed Output with System 1 and System 2 Thinking:*

The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.

[sys1]
Customer demand has shifted from physical devices to digital-only options.
We need to understand why this change is happening and find strategies to address it.
Key factors may include cost, convenience, and technological trends.
[end sys1]

[sys2]
**2.1 Understand the Problem:**
The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.

*Reflection:* 
"I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"

*Creative Perspective:* 
"Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"

**2.2 Analyze the Information:**
There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.

*Reflection:* 
"Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"

*Creative Perspective:* 
"Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"

**2.3 Generate Hypotheses:**
1. Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
2. There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
3. Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
4. Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
5. Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
6. Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
7. Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
8. There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
9. Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
10. Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)

*Reflection:* 
"Have I considered all possible influences? Are there any surprising factors that could explain this shift?"

*Creative Perspective:* 
"Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"

**2.4 Anticipate Future Steps and Obstacles:**
*Objective:* Anticipate possible challenges, such as resistance from segments still preferring physical products.

*Reflection:* 
"What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"

*Creative Perspective:* 
"Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"

[end sys2]

[sys1]
To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions.
Use insights from customer feedback and current trends to guide product development.
Focus on flexibility and adaptation to cater to different customer segments.
[end sys1]


There seems to be something of an optimism outbreak on after the London conference last week; here's McChrystal, saying he expects that by December, a significant chunk of the Taliban will have gone quiet - notably, he seems to expect a lot of semi-supporters to jack it in quietly. Well, two Friedman units, eh. A significant tribe has apparently agreed to sign up with the Afghan government.

Ahmed Rashid has a must-read on talks during 2009. He argues that the major diplomatic story in this is that the Saudi and Afghan intelligence services have developed their own contacts to the Taliban, bypassing the Pakistani ISI.

Obviously, if part of the strategy is to bypass the ISI as interlocutors, then it's going to be indispensable to compensate Pakistan on the other side of the table. As said here:

It would be a really fruitful thing for the Obama administration to start involving itself in an India-Pakistan peace process

However, this doesn't seem to be happening. You'd think the point that there are limits to what Pakistan can do without pushing public tolerance too far, and that they need time to consolidate after last year's fighting, would be obvious in the context of a "surge" based on counter-insurgency principles.

That's what the container terminal looked like this week (from here). Nathan Hodge of Wired has two good pieces, about progress reopening the harbour and bringing in a huge barge full of drinking water, and surveying the bottom of the harbour, which may not be in the same place any more.

Brett Holman of Airminded dropped in this link, with an excellent pic of the airport apron.

Meanwhile, we get links. I have to say I really don't know what to say about that suggestion, except that sending anything that needs refrigerating sounds no more helpful than sending anything that needs making up with water...

I'd also like to send the blog's good wishes to an old friend who's actually being deployed by their NGO.

Perhaps we shouldn't be so hard on the British wanktank movement. Here's an example of the US version. It turns out that the people responsible for a break-in at U.S. Senator Mary Landrieu's office, who apparently posed as telecoms company technicians in an attempt to "tamper" with the phones, were led by a character who was an "Undergraduate Fellow of the Foundation for the Defense of Democracies". He later acted as "assistant director" of a program at Trinity Washington University whose purpose was:

to introduce students in liberal arts colleges to concepts in intelligence studies and potential careers in intelligence

Is it out of place to note that the FDD is one of the stops on Alexander Melegreaou-Hitchens's World Tour of Wanktankery? And does anyone wonder if renta-quoting seems a tad dull compared to actually getting your G. Gordon Liddy on? Give him time; yer man also acted as the Operations Officer of a Department of Defense irregular warfare fellowship program, which is close enough to being a media "terrorism expert" for folk music.

I'm amused by the descriptions here and here; their defence is that they "wanted to know how they would react if the phones were inoperative". According to witnesses, they fiddled with a phone, called it or pretended to on a mobile device, and announced that they couldn't reach it, presumably in order to claim that there was a fault and they were there to fix it.

Strangely, for people with absolutely no evil intent, they seem to have replicated a 1950s MI5 bugging operation; declare there was a "fault" on the phones, then arrive posing as telephone engineers. Of course, the key element was that the real thing could ask the phone company to stage a deliberate fault, something this lot appear to have missed.

Ritterband's LinkedIn profile describes him as Communications Manager of the Conservative Party; he's also Boris's marketing director, and a major figure in Michael Howard's 2005 election campaign and David Cameron's leadership campaign. Which means that he's responsible for a significant percentage of the most vomitous public speech of the last decade.

Friday, January 29, 2010

No prime minister is indifferent to his or her legacy, and however much he feels stale controversies are being aired with little new public evidence, he knows tomorrow will be important for him, and his future public life as world statesman, Middle East envoy, spiritual healer and businessman.

Spiritual healer? As if the "world statesman" bit wasn't hilarious enough, or the "businessman", bit as opposed to "boardroom table ornament". The whole piece is sourced to "friends", British newspaper and especially Blairite code for "his PR people wanted to get this out", so presumably he actually believes this or at least tolerates his media advisers saying it.

Sunday, January 24, 2010

So, are the Americans really "prioritising foreign soldiers over aid" in Haiti? Thankfully, the national press tried to answer this question with facts. Well, not really. Spencer Ackerman and Laura Rozen actually asked intelligent questions rather than the usual "Two days after the giant earthquake destroyed all port facilities, critics asked why UN aid was still taking so long to arrive in the stricken region. After all, this 70kg journalist and his 88g sat phone got through just fine...anyone been raped and speak English?" stuff.

Apparently there are about 140 air movements through Port-au-Prince daily on average, of which 50% are allocated to NGOs and the rest to US and other government aircraft. The rate reached 200 on the Sunday following the earthquake; this is despite there being no radar or radio navigation aids since the earthquake.

A logistics system is a linear production process. Computer people would prefer to think of it as a loop construct. Therefore, the total capacity is determined by throughput - by the rate at which it loops. That, in turn, is set by the slowest element of the process.

In this one, goods are being loaded on aircraft, that then take off, fly to Haiti, land, unload, take off, and return. Now, the aircraft can leave from many, many different airports, so I think we can rule out that step as being the limiting factor. For landing, the minimum separation between planes is the limiting factor; the standard 3 miles horizontal separation is a minute's flying time at 180 mph (156 knots - the approach to Heathrow is flown to 4 miles out at 160kts), and the wake-turbulence separation for a heavy aircraft like a 747 is three minutes. So the maximum separation will be four minutes or thereabouts; a lot of the aircraft being used are able to slow down much faster on final approach and some are less heavy, so it's probably somewhat less.

One aircraft every three minutes on one runway gives us 20 movements an hour. If they aren't going to pile up there, they've got to leave at the same rate, so that's 10 in, 10 out, and we'd reach 140 movements in 7 hours. Considering that the landing lights don't work and the control is visual, they can't be far off operating at capacity. But that's still not the last word.

You may be able to land a plane every three minutes, but you probably can't unload it in three minutes. And you've got to return the empties, as well; you can temporarily up the rate by using more of the movements budget for landings, but eventually this will mean you have to stop to send aircraft back. The best performance is achieved by operating at the highest average capacity. So, the slowest part of the process is probably unloading and turn-around more generally.

This brings up another issue. If we start the day with 10 aircraft arriving an hour, that then spend 3 hours on the ground, 30 aircraft will be there before the first one leaves; we'll need at least 30 parking spots.

In fact, because unloading is the most restrictive step in the process, it's optimal to always have a queue. Otherwise, there will be moments when the most scarce resource in the whole thing - a forklift truck and its driver - will be waiting for cargo to move, at which point we're operating below capacity and wasting time.

In practice, this will be the operational limiting factor; it doesn't matter if a plane has to wait to unload, but it does if the next one can't leave the runway and the next one has to go-around and divert. So, we've arrived. Our limiting factors are forklifts and parking.

The USAF air traffic controllers announced a maximum two-hour turnaround on the night after the earthquake, and further insisted that all aircraft arriving in the airfield circuit have enough fuel to go to their destinations without refuelling, there being (obviously) none to spare in Haiti.

You get 11 hours of daylight there at this time of year, and presuming that it's still visual-only, that's 12 movements an hour, six in, six out. That's about 0.25 Heathrows. With a maximum turnaround of two hours, this implies that there are 12 parking spaces available on the ramp. That should also explain what happened to that MSF flight; apparently another aircraft went technical on the ground, blew its two-hour slot, MSF had to go-around, and David Aaronovitch and Noam Chomsky were at once united in blowhardry, not for the first time.

Actually, come to think of it, you can get the NOTAMs for Port-au-Prince by going here and searching for MTPP. It turns out that an instrument flight plan is mandatory...just like at Heathrow, and they want to know how much weight, how many items of rolling stock (!), and how many passengers you have. Oh, and:

Major offshore petroleum discharge systems (OPDS) components are: the OPDS tanker with booster pumps and spread mooring winches; a recoverable single anchor leg mooring (SALM) to accommodate tankers of up to 70,000 deadweight tons; ship to SALM hose lines; up to 4 miles of 6-inch (internal diameter) conduit for pumping to the beach; and two BTUs to interface with the shoreside systems.

Rather less depressing; Wired reports on the array of open-source IT tools for disaster relief getting their first use in earnest in Haiti. I remember when your main source for things like Google Earth overlays of aerial photos was Kathryn Cramer, and that was in the United States. However, there's something I saw that wants drawing attention to.

They've already got that, but "faster" only in the sense that it's already done... They're limited to a few STM1s, which were quickly overwhelmed by the relief workers. This is a common problem in disaster relief, we saw it particularly when we were working in Indonesia and Thailand during the tsunami... An area that had quite modest Internet usage, and infrastructure which may not be great, but is sufficient to its present requirements, gets a flood of relief workers in who all want to use Skype simultaneously, and determine that the perfectly-functional and previously-sufficient Internet is "broken" and needs to be reengineered.

The existing chain of microwave relays is the Haitian ISPs' fix for the problem of Teleco having a monopoly fiber landing and setting astronomical prices on access to it.

I'm not interested in reengineering anything, but I am interested in making sure that if aid money goes to the incumbent to fix their fiber, at least the community gets something out of it in the form of the monopoly being broken. Otherwise the fiber being fixed does no one any good, because they still won't be able to use it, same as before the earthquake.

It's very easy to spend money and make things worse than they were before

He's referring to the Haitian submarine cable landing, which was destroyed, although the fibre itself may still be present, and the fact that they did have alternative connectivity to the Dominican Republic by microwave link. I do like the point about relief workers with MacBooks (and corporate preening PR men back at headquarters pressing for teh videos for the nine o'clock news) as a denial-of-service attack, however.

The NANOG community has been helping in various ways, including by finding ways for the engineer in charge of their NAP to get his family out of the country, diesel for the backup generators, and such.

Well, this is hardly surprising; the FBI was in the habit of pretending to be on a terrorism case every time they wanted telecoms traffic data. Their greed for call-detail records is truly impressive. Slurp! Unsurprisingly, the lust for CDRs and the telcos' eagerness to shovel them in rapidly got the better of their communications analysis unit's capacity to crunch them.

Meanwhile, Leah Farrell wonders about the problems of investigating "edge-of-network" connections. Obviously, these are going to be the interesting ones. Let's have a toy model; if you dump the CDRs for a group of suspects, 10 men in Bradford, and pour them into a visualisation tool, the bulk of the connections on the social network graph will be between the terrorists themselves, which is only of interest for what it tells you about the group dynamics. There will be somebody who gets a lot of calls from the others, and they will probably be important; but as I say, most of the connections will be between members of the group because that's what the word "group" means. If the likelihood of any given link in the network being internal to it isn't very high, then you're not dealing with anything that could be meaningfully described as a group.

But there's a really huge problem here. The modelling assumptions are that a group is defined by being significantly more likely to communicate among itself than with any other subset of the phone book, that the group is small relative to the world around it, and that it is boring; everyone has roughly similar phoning behaviour, and therefore who they call is the question that matters. I think these are reasonable.

The problem is that it's exactly at the edge of the network that the numbers of possible connections start to curve upwards, and that the density of suspects in the population falls. Some more assumptions; an average node talks to x others, with calls being distributed among them on a well-behaved curve. Therefore, the set of possibilities is multiplied by x for each link you follow outwards; even if you pick the top 10% of the calling distribution, you're going to fall off the edge as the false positives pile up. After three hops and x=8, we're looking at 512 contacts from the top 10% of the calling distribution alone.

In fact, it's probably foolish to assume that suspects would be in the top 10% of the distribution; most people have mothers, jobs, and the like, and you also have to imagine that the other side would deliberately try to minimise their phoning or, more subtly, to flatten the distribution by splitting their communications over a lot of different phone numbers. Actually, one flag of suspicion might be people who were closely associated by other evidence who never called each other, but the false positive rate for that would be so high that it's only realistically going to be hindsight.

Conclusions? The whole project of big-scale database-driven social network analysis is based on the wrong assumptions, which are drawn either from military signals intelligence or from classical policing. Military traffic analysis works because it assumes that the available signals are a subset of a much bigger total, and that this total is large compared to the world. This makes sense because that's what the battlefield of electronic warfare is meant to look like - cleared of civilian activity, dominated by one side or the other's military traffic. Working from the subset of enemy traffic that gets captured, it's possible to infer quite a lot about the system it belongs to.

Police investigation works because it limits the search space and proceeds along multiple lines of enquiry; rather than pulling CDRs and assuming the three commonest numbers must be suspects, it looks for suspects based on the witness and forensic evidence of the case, and then uses other sources of data to corroborate or refute suspicion.

To summarise, traffic analysis works on the assumption that there is an army out there. We can only see part of it, but we can make inferences about the rest because we know there is an army. Police investigation works on the observation that there has been a crime, and the assumption that probably, only a small number of people are possible suspects.

So, I'm a bit underwhelmed by projects like this. One thing that social network datamining does, undoubtedly, achieve is to create handsome data visualisations. But this is dangerous; it's an opportunity to mistake beauty for truth. (And they will look great on a PowerPoint slide!)

Another, more insidious, more sinister one is to reinforce the assumptions we went into the exercise with. Traffic-analysis methodology will produce patterns; our brains love patterns. But the surge of false positives means that once you get past the first couple of hops, essentially everything you see will be a false positive result. If you've already primed your mind with the idea that there is a sinister network of subversives everywhere, techniques like this will convince you even further.

Unconsciously, this may even be the purpose of the exercise - the latent content of Evan Kohlmann. At the levels of numbers found in telco billing systems, everyone will eventually be a suspect if you just traverse enough links.

Colonel Marchpole's department was so secret that it communicated only with the War Cabinet and the Chiefs of Staff. Colonel Marchpole kept his information until it was asked for. To date that had not occurred and he rejoiced under neglect. Premature examination of his files might ruin his private, undefined Plan. Somewhere, in the ultimate curlicues of his mind, there was a Plan.

Given time, given enough confidential material, he would succeed in knitting the entire quarrelsome world into a single net of conspiracy in which there were no antagonists, only millions of men working, unknown to one another, for the same end; and there would be no more war.

Want a positive idea? One reading of this and this would be that the failure of intelligence isn't a failure to collect or analyse information about the world, or rather it is, but it is caused by a failure to collect and analyse information about ourselves.

A proposal to deal with linkspammers - set up a central blackhole where you send everything that you spam-rate, and use the feed of URIs from it as an input to your automated spam filter. Its own web page is here. Unfortunately, as I point out, there's a serious flaw here.

Basically, if we're going to filter out anything containing links that have already been reported as spam into a feed that's open to the world at large, we've created a means for anyone to censor anything, across the whole 'sphere. To make it happen: generate spam comments, deliberately spammy spam comments that will be immediately recognised and deleted, but include the target URI as part of the payload.

Spammers have been using text taken from old books, or harvested from the Web at random, in order to fool spam filters for years; more recently, they've taken to harvesting text from the site they're spamming, and including that with the spam, so as to fool the spam filter. So this doesn't change existing practice or code very much. And adding legitimate links might well boost your chances of getting the commercial payload through; you'd have to think carefully about whether this was a good or a bad thing, as this attack will work best if the spam is detected and submitted automatically.

Anyway, we run off thousands of spam comments containing links to - say - Sourcewatch, RealClimate, or whoever, as well as links to enralgemypen1s.tv. The spam filters sweep them into the distributed filter feed. Now, anything containing those links is banned from any site that uses the feed to prime its spam filter; and, of course, once one site's filter starts automatically sweeping them up, their concentration in the crapfeed will only go up.

Part of the problem here is not looking across the layers; we've already had this problem in internetworking, where various schemes to filter out attack traffic dynamically have foundered because the enemy realised that if they could get IP packets routed with arbitrary source addresses, they could also set real source addresses, and further, they could use this not just to escape blackhole routing, but to have whatever consequences that followed sent straight to somebody else. In fact, if you did this to something like a group of anycast DNS servers, you could generate truly epic volumes of traffic heading for the target, traffic that came from sources that they couldn't possibly blacklist.

For example, one IP abuse problem is bogon packets - ones that come from or go to networks that shouldn't be in the Internet Routing Table, because their addresses aren't allocated to anyone or they're reserved for private or special purpose use. These are a problem partly because this means that more than one network in more than one location could be under the same address, and therefore that bad things could happen with the routing protocols, and partly because squatting in bogon space is one way of avoiding responsibility for stuff you send, like spam, malware, denial-of-service attack traffic, etc. Fortunately, you can set your router filters to deny anything from or to any of the networks in the bogon feed provided by Team Cymru and forget all about it.

This works, however, because the bogon feed uses a whitelist approach based on the lists of assigned networks prepared by IANA and the regional registries. (Can anyone guess where I got the idea for the Vfeed from?) Out of the totality of possible IPv4 networks, it subtracts everything that's been released for use - what's left is the possible bogon space. If it received the contents of other networks' bitbuckets, you could bet that someone would hook up a BGP session and inject the whole of Google's address space, or worse, the whole of Level(3), Akamai Technologies, or the London Internet Exchange's, and watch half the Internet disappear from their route view and everybody else's like blips off a radar screen.

Fortunately, if it was prepared in that way, nobody competent would dare use it, and the incompetent usually don't bother filtering bogon packets...which is why they exist.

Similarly, e-mail spam filters used to send warning messages back to the originating user, until it was noticed that you could set entirely spurious Reply-To fields and deluge all kinds of other people with crap that they couldn't blacklist because it came from legitimate mail servers.

But the lesson seems to need re-learning quite a bit; explosives detection would seem to be a field full of promise, both in the negative version of the attack (you spray around the smell, and then you smuggle the explosives, because the increased concentration of nitrates from them isn't detectible against the background contamination) and in the positive version (you spread the smell around the passengers, so they trip the detectors, causing havoc, and eventually causing them to turn the detector sensitivity down - and then you smuggle the explosives). Any attempt to achieve security by comparing a target stream with another depends on the independence of the two streams, just as any attempt to increase bandwidth by combining two parallel streams depends on their independence.

Come to think of it, I'm slightly surprised that I haven't seen David Cameron poster spam, as that's a legitimate source that generates content from anonymous Web input.

Thursday, January 21, 2010

Metafilter! I love you all. I needed a crowd, and you brought me a mob. It's been a four-figure day for the blog by 10 a.m., and more importantly created a truly superb boom in Dave from PR remixes. Looking at the detailed server log from TYR Classic, loads of people are googling for things like "blank david cameron" - a telling comment in itself - and "cameron poster generator". You will be infected. It's worth noting that it's not just posters; I am aware of three poster generator sites - Andy Barefoot's, the Cameronizer, and MyDavidCameron.com, whose domain name is a minor classic in itself.

Lessons? For a start, the Vital Importance of Stable URIs and REST. When Andy deployed the feature that produced a stable URI for your poster, there was an explosion of creativity. Two reasons - first, you could show them off, share them with others, promote other people's work. Links introduced the social dimension. Secondly, it got rid of several steps in the process - download the image, re-upload it to flickr/somewebhost/whatever, then pass it around. Making it a Web service gave the user instant results.

Less technically, the poster and its fate tell us something about Dave from PR. The whole project of Cameron is essentially a drive to implement all the most satirised features of Tony Blair; they're working on the theory not just that the 1997 campaign was an effective model for getting elected, but that the public actually likes media manipulation and verb-free sentences and trust-me face and truthiness and faintly Jesus-y canned emotion. It's not the ad campaign - it's the product.

As a result, everything about him has the odd over-perfectly stylised quality of that poster, and is therefore permanently poised on the edge of being self-satirising. It may be our last best hope to push him over it at every opportunity.

Today, I love all the people. (Although, so did Erich Mielke. So don't kid yourself I'm going soft.)

Wednesday, January 20, 2010

Somehow, that poster seems almost designed for satire. There are excellent reasons why it works so well; it's possibly the most stylised example of a political advert I can think of. In a sense, it's a movie - not at all original, but highly competent in a limited way, and therefore a perfect subject for parody. You only need to identify a small number of controls, or variables, that define it, in order to produce a message that matches the requirements of the format perfectly but has an entirely different payload.

J.G. Ballard, of course, was very much aware that display advertising is in some ways a programming language. Hack work is one of the standard literary experiences, but Ballard's time as an ad copywriter must have been especially telling on his writing. Ballardian has a superb post on his 1960s project to create a range of content-free adverts, based on randomly cut-up texts and unrelated photos, that he placed in Vogue.

Look at either the original, or the skits; note carefully where the content is. The backdrop is soothingly grey, but not blank - it's chosen to be content-free but without being actually blank or being a block colour. Blank space or block colour are visual statements - in modernism, you're being asked to concentrate on the elements of the object you'll actually interact with, in post-modernism, you're being asked to project your own internal imaginings onto the blank space. Either way, if you make the colour field bright red, you're putting the viewer on notice that you want to say something. The blurred-out background of the Cameron posters is the colour of nothing.

In front of it, we've got the heavily retouched Dave. Look where he is. User-interface research in computing suggests that the most important part of the visual for the majority of people is to be found as follows; divide the screen in four equal quarters, then divide the top left-hand one in quarters again, and pick its lower right-hand sector. Search engines assume that over 90% of clicks land in this zone on the first page of results. (Back in 2004, ignoring this was how I did the Viktor Bout story - just keep ploughing through the Google output.)

So the big pink face goes here - it acts as a graphical and thematic anchor for the eye. Thinking of the poster as a frozen movie, the action begins here. It's also true that we're likely to pick out the monkey in the background flow of images first - before we react to anything else on the poster, we have the chance to feel the tebbly-tebbly concerned smile at a subrational, sublinguistic level.

We move on; saccading from left to right and top to bottom, the next scene is the message in big friendly letters, as Douglas Adams would say. It's worth noting that the real thing always has two sentences, and although they are united by the same typeface (Franklin Gothic), the real poster has a slightly different colour mask for the second. This signals that there is a plot relationship between them. On the original poster, Cameron promises a crisis about the budget in the first colour, then promises not to cut the NHS budget in the second. So we're setting up conflict and resolution here.

No matter that the two statements are contradictory - in fact, if they weren't, it wouldn't work as a film. We move on southeastwards - first of all, we see the whizzy logo, so we know how to recognise the next element in the plot, and then, we get the pay-off, the strapline at the bottom right-hand corner of the poster. This is important - it's the finale, and it's got to contain something actionable, in the intelligence sense rather than the legal sense.

For the first time across the vast span of three or so seconds we've spent watching this drama, we see the word "CONSERVATIVE".

THE END.

It's probably worth remembering that a lot of these are meant to be installed next to motorways or major rail routes, where we will in fact approach them at speed. Treating it as a film rather than a static artwork is therefore very appropriate.

Saturday, January 16, 2010

There's quite a lot of buzz about this story, in which a DHS report into criminal use of aircraft over the South Atlantic gets rehashed. The "Air Cocaine" case in Mali has given the whole thing another layer of sexy, of course, and it's good to know that the problem is recognised - even better that it's no longer considered to be a potential ally.

However, it's still a subject on which governments project their existing prejudices. For example, it's not apparently enough for there to be 10 tons of cocaine in the 727 - to get anyone's attention, you need to get a terrorist in there too. Similarly, you rarely get away without a ritual attack on Venezuela, which is getting to be a sort of happy hunting ground for fans of state sponsorship theories like the Bek'aa Valley used to be for Dick Cheney. And, of course, there's the temptation to look for anything that connects the story with Viktor Bout.

Of course, the main reason why such aircraft might pass through Venezuelan airfields is that it's on the way; 727 serial 21619 stopped in Fortaleza, which is even closer to West Africa, on the way out and probably on the way back, but I suppose Brazil is too big to pick on. The report linked does at least note that the geography is important.

For people like Paul Wolfowitz and his "network of friendly militias", I suppose they saw a provider of useful services. The drugs people see it as part of the Drug War. The arms trade people see it as a small arms transfer issue, and the terrorism people see it as something to do with terrorism. I'm trying to see it as something to do with the ambiguities of globalisation; in a sense, it doesn't really matter which terrorists or whose arms are travelling in whose aircraft.

There is, however, a fringe economy that empowers and profits from all these things, and there's the rub. It does so in ways that confound the aims of the powerful (like the drugs and the terrorists) and it does so in ways that further them (like the Iraq logistics and arms to Angola). Finding convenient terrorists shouldn't be necessary.

One thing that interests me about the South Atlantic element of this is that, if the Viktor Bout experience is anything to go by, a critical element is hybridisation with the legitimate economy, and especially major nodes of trade.

Both the Sharjah Airport free zone, for example, and the UAE airports themselves essentially permitted Viktor Bout, and many others, to operate outside the law while also enjoying the facilities of civilisation. They could get major aircraft maintenance done, compete for legitimate cargo, and also stash planeloads of arms in bonded warehouses. A long runway is a necessary but not a sufficient feature; there was a reason why they didn't set up camp in Riyan or Machiranish.

So where's HQ for the West Africa/Latin American community? I still like Ajay's suggestion of Conakry, especially in the light of its increasingly dysfunctional junta (although, the trick is to put your base well away from the customers...). But I would expect to see more traffic there on the Vfeed. However, it's quite probable that it will be located somewhere where there is an active interface between extreme free markets and an authoritarian state, and where there is substantial infrastructure. In fact, you could almost identify free zone authoritarians as a subtype of the modern thinkers.

Note that the typical aircraft types in the Atlantic are Western - Gulfstreams and Boeing 727s. This has consequences for their maintenance and support.

We formed investigative teams composed of one international and one Liberian investigator. Together they handled individual cases, traveling to a recruit's home village to verify data and garner character references. We compiled and assessed existing public records for accuracy and volume and ran candidates' names through the limited records that we found credible. To our surprise, some of the best records came not from the government but from local NGOs such as the West African Examination Council, which had administered and kept records of high school achievement tests for decades.

The Examination Council. There was a functioning exam board in 90s Liberia; that's absurd and heroic all at once.

OK, so I did two things - I upgraded to OpenSUSE 11.2/KDE 4.3, which is great, and I've installed SQUIN, the semantic Web query server, on my laptop in order to work on WhoseKidAreYou. The concept of SQUIN is that it provides a SPARQL end point to do queries over the various, interlinked sets of data that conform to the Linked Data standard.

So, I should be able to pull data from the FOAF db, from DBpedia, and all sorts of other stuff in the same query statement. Cool. And you've got to hand it to them, as well, the install is almost comically easy. But, as with SPARQL in general, there are things I'm not getting. The idea of Linked Data is that you should be able to follow links from a record retrieved from one DB into another related one - for example, if the DBpedia record for somebody contains FOAF information, the query client should note the link, recurse along it into FOAF, and get you any information that matches your query that's in FOAF as well as DBpedia.

You'd think that the main problem would be constraining the search and filtering the results. Essentially, I'm trying to replicate the behaviour of a cynical and intelligent person searching the Web for the authors of everything they read, and it's obvious that someone doing that uses most of their brain effort to sieve the search results. Similarly, if you're writing a SQL query to pull data out of a classical relational database, your biggest concern is usually how to filter, reduce, group, aggregate, summarise, or limit the volume of data that comes back.

But I find the difficult bit with SPARQL is maximising the volume of data that comes back. It's incredibly easy to get nothing at all for quite trivial queries. Another thing is that if one of the variables in the query doesn't match, none of them do, and the query will return nothing. You can use the OPTIONAL keyword, but as far as I can see, you need to OPTIONAL each and every statement. The syntax is annoyingly "almost, but not quite, entirely unlike SQL" and it's oddly difficult to get a data variable, rather than a URI, into your query.

Also, I find the Linked Data element of this a little hard to visualise. Presumably, if you want to query across datasets, you need to use prefixed namespaces that are common to them all. I think, but I'm not sure, that you can mix multiple prefixed namespaces.

Regarding SQUIN itself, I'm also suspicious that the queries return very, very fast; there's not enough time for it to be doing any recursing that involves multiple network round trips. Here's an example:

This should declare the query variables in the SELECT clause, get the value of the Person/Name property of the DBpedia article Martin_Amis, bind it to ?name, then get all the values of the Person/influencedBy property that match ?name, bind them to ?influenced, and then the FOAF:Page values that match ?influenced. We're then, going to query FOAF for the FOAF:Knows values for each of the influenced, and their home pages.

As that's uncertain as to whether they have them, it's an OPTIONAL clause, as is the one that gets the foaf:pages in the first place. DBpedia's SNORQL interface chokes on the reference to Martin Amis (who wouldn't). SQUIN considers it valid SPARQL, but produces no results whatsoever. If you browse over here, you'll find that all the values involved are present and as described; and, indeed, the first influencedBy has a foaf:page attribute. In general, semantic web things seem to be good at failing to return data they actually have under the attributes they have for it.

What is it that I'm missing? Is there a huge tarball of data I need to load in SQUIN? Surely the point of Linked Data and semantics is that you don't have to scrape the Web and snarf it all into a big database, but rather treat data on Web sites as if it were in a database?

An interesting question. Donald Rumsfeld and Dick Cheney were survivors of the Nixon and Ford administrations. Which crawling horrors from the Bush years will be around to plague us in the future? With a year's hindsight, it ought to be fairly clear who's recovering. For example, a lot of the senior posts in the current shadow cabinet are held by people who were in it within the first couple of years of Blair, and you could say the same for punditry.

The question is easier to answer in the negative - it's pretty unlikely that Douglas Feith or Dan Senor have glittering bureaucratic futures ahead of them. A whole gaggle of second tier people were killed off by the various scandals ranging from CIA corruption to the US Attorneys. The connoisseur's answer is David Addington, but there's only one of him. That leaves us with an impressive known unknown - we know there will have been a handsome new crop of pig-bastards, but we don't know who they are.

Apparently a lot of them were transferred into the permanent civil service in the last months of the regime; presumably, this involves gazetting the appointments, so there must be data out there somewhere. It's still an astonishing thought that they managed to keep the Vice-Presidency's phone book secret.

Sunday, January 03, 2010

The Obscurer's coverage of the Undabomber has been marked by one man. Here he is:

Peter Hoekstra, the senior Republican on the House intelligence committee, said it was examining Mutallab's links with the radical Yemeni imam, Anwar al-Awlaki, who has inspired a number of terrorists.

Awlaki had contacts with Major Nidal Hasan, the Army psychiatrist who is accused of carrying out the massacre at Fort Hood, Texas, in November in which 13 people were murdered. According to government officials, Awlaki was also the spiritual adviser to two of the 9/11 hijackers, Khalid al Mihdhar and Nawaf al-Hazmi, while he was an imam at a mosque in northern Virginia. The FBI investigated him in 1999 and 2000, believing him to be a possible procurement agent for Osama bin Laden.

In Toronto, a terror cell watched videos of Awlaki at a makeshift training camp where an attack was planned on the Canadian parliament and prime minister. "He's a star attraction as a recruiter to young Americans and Canadians," one former American intelligence official told the US media.

This month, in an interview with Al Jazeera, Awlaki expressed surprise that the US military had failed to uncover Hasan's plan, to which he gave his backing. "My support to the operation was because the operation brother Nidal carried out was a courageous one, and I endeavoured to explain my position regarding what happened because many Islamic organisations and preachers in the west condemned the operation," he said.

Awlaki left the US and moved to Yemen in 2002 where he established an English-language website that has thousands of followers around the world. In January 2009, he published an online essay, 44 Ways to Support Jihad, in which he asserts that all Muslims must participate in jihad, whether in person, by funding mujahideen or by fighting the west.

There's something missing here...can you spot it?

Concerns about his influence in the UK have been expressed by experts on community cohesion. In August, the Observer reported anger that Awlaki was due to speak via a video link at Kensington town hall. The broadcast was dropped after the local council stepped in. He has also been invited to give talks via video link at several London universities. "Mutallab is the latest in a long list of terrorists [Awlaki] has inspired and encouraged," said Alexander Meleagrou-Hitchens of the Centre for Social Cohesion.

"The preacher has long been a highly respected figure within a number of British university Islamic societies because, unlike most other radical preachers, Awlaki speaks English as a first language, and being born and raised in America has given him a good understanding of western culture. This makes him very appealing to young western Muslims."

Meleagrou-Hitchens called for British universities to increase their vigilance. "This incident should act as a wake-up call to university authorities," he said. "It is crucial that they now accept the central role they must play in resisting extremists and preventing student groups from promoting hate preachers."

Did you spot it? The Obscurer didn't actually say that he had any connection with the pants bomber. They didn't even quote Hoekstra saying so - and Hoekstra is a comedy rightwing buffoon anyway. They didn't adduce any evidence of his connections with him in any way - just cut straight to Alexander Meleagrou-Hitchens. Whose kid are you?

PolEx's Web site has an "Alumni" page, but mysteriously it bears no trace of him. Google, however, knows:

He has also worked at the Stanford University based think tank, the Hoover Institution for War, Revolution and Peace, and the Washington DC based think tank, Foundation for Defence of Democracies (FDD).

He holds an MA in International Relations from Brunel University, and a BA in Classics from King’s College London.

Alexander researched for publications providing policy recommendations on creating a robust defence against the threat of terrorism in the UK and abroad.

FDD as well! Free cuppa for you! There is, of course, no suggestion of or link to any work on terrorism he ever did.

Recordings of Anwar al-Awlaki, an al-Qaida sympathiser who is believed to have inspired Abdulmutallab in Yemen, can be bought through British-based websites and bookshops. Three shops in London and Manchester were contacted by this newspaper last week. Staff said they could sell DVDs of the speeches by the cleric, who is banned from the UK.

As recently as last April, students at London's City University Islamic Society's annual dinner were invited to hear the words of al-Awlaki being broadcast live into Britain.

So why is he "believed" to have inspired pants boy? Where is the evidence? It's not even the electioneering torture fan Hoekstra this time.

Alexander Meleagrou-Hitchens, a research fellow for the think-tank the Centre for Social Cohesion,

For it is he.

said that al-Awlaki has become an increasingly influential figure. "For well over a year now, organisations such as ours have repeatedly warned about the dangerous influence of this man, with most of our warnings falling on deaf ears," he said.

Call now and buy your anti-terrorist water ioniser - 20 per cent off before the end of this broadcast! And don't forget to donate now and claim Gift Aid!

"They had no objection to his giving a video sermon to a gathering at Kensington and Chelsea town hall. We are also often told that, although al-Awlaki's views may be unsavoury, he has never been convicted of any crime. Clearly, this excuse is simply not good enough."

The excuse that he hasn't done anything wrong.

Further, Hitchens Minor seems to be missing someone in his laudable crusade on the home front. I refer, of course, to the current and past tenants of Kensington & Chelsea Town Hall, or in other words, the Conservative Party in London. Could this perhaps have something to do with the fact that his boss at Policy Exchange is now the Conservative Mayor of London's director of policy?

In fact, in a sense, they do recognise it as a cat - it's roughly the right size and shape, it's in the right place, and if you wave a piece of string in its whiskers it responds much like a cat. But actually, as the project Web site tells us:

The robot was designed to reproduce the behaviour of rats as they use their whiskers to explore their environment. To get a clearer picture of how rats use their whiskers we filmed them using high speed video cameras (500fps) and manually tracked the position of each whisker in the array on a frame by frame basis. Software based automatic tracking is still very much in its infancy though there are a number of groups (including our Sheffield partners) who are now working toward such an application.

The data from this whisker tracking allowed us to quantify the kinematics of whiskers as the rats explored novel environments. From this we found that following a whisker making contact with an object there was a very rapid (~13ms) change in the velocity profile of the ‘whisking’, or movement pattern of the whiskers. We also observed that the rat will tend to move, or orient, its nose toward the exact point of contact.

Our hypotheses were that the rat was trying to optimise the force applied by the whiskers making contact with the object as well as bringing as many addition whiskers as possible, and its nose for smelling, to bear on that point. The orienting behaviour we see as an example of a higher level control loop through the brain, very similar in nature to how we as visually dominant animals rapidly orient, or saccade, the fovea of our eyes toward interesting events detected by our peripheral vision.

To this end we designed our robot to mimic both the low level contact mediated adaptation of the whisker motion pattern and the ability to orient its ‘nose’ towards points in three dimensional space. Designing the physical robot to be capable of mimicking these behaviours allows us to test different computational models of the underlying brain structures which can control it.

So it's a ratbot, and interestingly enough, it's an example of a hardware simulation of a biological phenomenon.

But this is also an example of an interesting design phenomenon; if you want objects to be immediately comprehensible, it helps to use the patterns Dan details here; notably, in this case, similarity, mimicry, and role-playing. Everyone knows how to act around a cat; a robot, not so much. We've been trained to do so by cats. And this part of the project will be unavoidably cattish:

We hope to be able to demonstrate the validity of the proposed brain model by the robot being able to chase an object (perhaps a remote controlled car) moving through its whisker field

Aww. Also, our associations for cat- and also dog-like behaviour stimulate our curiosity towards them, in part because we project it on them. Just as making a new smartphone an interesting object to handle speeds the learning process, a robot that encourages curiosity and interaction towards itself will speed its users' learning process.

I suspect that the response to the Bristol Scratchbot would have been rather different had we been told in advance that it was emulating a rat. (Special note - much of the robot was made on a rapid prototyper.)

an online platform that enables us to tap into the wisdom of crowds to resolve difficult policy challenges

Of course, it might be possible to make statements about this if only it was better specified. So everyone's contented themselves with making fun of the press release and blaming blogs for the Iraq war. Or something. But underspecified is good in some ways...

If you want a proposal that could certainly be delivered for less than £1m, I've got one. Free Our Bills, MySociety.org's proposed Web site to allow anyone to track the drafting process of UK legislation - to view amendments, make notes, monitor changes, and lobby Parliament in real time. Here's my crack at a rough design of it, with a link to the Germans' solution to the same problem.

Essentially, it's a package manager for legislation - this talk at CCC describes relevant technology that's actually being used for the Government's data.gov.uk.

And you know what? David Cameron is on record supporting it. However, his party is also keen to fuck up anything the new Speaker, John Bercow, who is also sympathetic and is actually in charge of these things, wants - purely out of spite.

Also, if there's any change left out of the cool million after that, you could even make a start on Who's Lobbying - specifically, by setting up an internal registry of contacts with lobbyists for the Government. I asked Tom Watson MP about this last OpenTech - he thought there was no such log, which surprised me as it is something that would materially assist No.10 Downing Street and ministers in tracking what is actually happening in politics, quite apart from its utility as citizen technology.

Whilst we're on the subject, a couple of other terrorism things. Kingpin of Comedy Gladio, intimate of Patrick Mercer MP, and self-made spy Glen Jenvey has been arrested, ironically on charges of inciting hatred against Jews. He's finally convinced somebody that he's a real jihadi! Not that he's likely to convince anyone else of anything at all...except Tories. The government must, indeed, look at these coincidences!

Meanwhile, in this season of prodigies, Ed Husain astonishingly says something quite sensible.

In both Britain and America demands for profiling all Muslims at airports are increasing in volume. This mindset not only fails to understand that most Muslims around the world detest al-Qaeda, but this outlook also cannot comprehend how terrorists are always one step ahead of the game. If it is Muslim-sounding names that are to be stopped, would a name like Richard Reid – the infamous shoe bomber – have been detected? If it is Asian men that are to be stopped, then we will see an increase in white men recruited for terror?

After all, al-Qaeda's English spokesperson is Adam Gadahn, a white American. If it is men who are stopped, we will see women terrorists emerge. Let us not forget Palestinian groups' repeated use of single women as suicide bombers. Do not underestimate the power of terrorists to recruit serving airline pilots and other aviation personnel. Where there is a will, there will always be a way.

The profiling of ordinary Muslims not only opens other avenues for al-Qaeda, but results in the harassment and potential loss of support from the very people we need on our side to contain al-Qaeda: ordinary Muslims.

He also spends quite a lot of time romanticising Sufism, retailing whines about Obama bowing to the Saudi king, and saying things like this:

Nearly a decade after 9/11, when compared with military budgets, where is investment in these soft-power, counter radicalisation projects? The silence says it all.

Indeed it does, indeed it does. Strangely, he didn't include a PayPal DONATE button. But it's a start, I suppose, and we should reward good behaviour when we see it, even from the wanktank community. We Are Reasonable People, after all.

It is easy to imagine that the authorities at UCL took quiet pride in the fact that they had a radical Nigerian Muslim running their Islamic Society. You can’t get more politically correct than that? They would therefore have had little interest in monitoring whether he was using a British university campus as a recruiting ground for al-Qaeda terrorists such as himself. The authorities at UCL should hang their heads in shame – or better than that, perhaps they should resign?

It may be easy to imagine that, but what are your offensive imaginings doing in something that claims to be a newspaper? Good job you only imagined it or else you could have been sued. It's a bit like adumbrating that way.

As you'll probably have guessed, I'm not terrified by the pantsbomber. As Spencer Ackerman puts it:

it doesn’t do any good to blow this out of proportion, since blowing things out of proportion to spur an overreaction is Usama bin Laden’s explicit strategy

However, as more details have filtered out, I've revised my original view of it a tad. My first thought was that it was another very low-grade terrorist - probably a self-starter, with limited contacts, shaky technology, and worse execution. Therefore, I thought, there was a strong case to be made that he was a sign of terrorist weakness rather than strength. This is the best they can do? Pants.

With more information, however, it looks like he wasn't quite as crap as that. On the other hand, everyone's talking about his using a charge of PETN; if true, that would imply he had a contact for valid high explosive. But the fact he went fut rather than bang argues against this. PETN is the most sensitive of military explosives with the exception of nitroglycerine - it's the stuff in detonator cord (Akzo-Nobel PDF - probably not one to read on the plane) - and as a result, it's quite rare. Nobody wants explosives that explode when they shouldn't. Had the main charge consisted of PETN, he should have exploded. (From the data sheet: NEVER attempt to cut by abrasion or blow with a sharp object!)

That, and the fact he was apparently seen fiddling with a syringe, suggests that this was yet another attempt to make the explosives at run-time as it were - as this comment at Bruce Schneier's explains.

Fortunately, doing serious chemistry in your chuddies is not the way they do it at Bayer AG; a special problem is that the reaction he may have been aiming for is exothermic enough that the stuff (and him) would have caught fire before the job was done. If there was a detonator of some sort present, that might have gone up in the heat and panic, thus accounting for the small bang reported.

Come to think of it, a serious barrier to this plan would be enduring a powerfully exothermic reaction involving a strong acid in your crotch long enough to initiate the explosion...also, it seems impossible to avoid making a spectacle of yourself such that someone would spray your blazing boxers with an extinguisher and/or bash your head in with a crash axe long before you were ready to explode. As Bruce Schneier points out, bringing the explosive and the detonator on separately would be hard to detect...but it's not clear that this is what pants boy actually did.

So, what does this tell us about the terrorist threat? I would argue I wasn't too far out to begin with. This wasn't a pathetic attempt, but it was still a weak, one-man job with poor technology and worse execution - just one of a slightly better class than we've had recently. And the startling bit is that far from being a self-radicaliser with a copy of the Anarchists' Cookbook and the York Notes to the Holy Koran, he claims to have actually been in contact with real live Al-Qa'ida members. To begin with, I was dubious of this - he might have meant that he did it with the same aims as Al-Qa'ida, or that they inspired him to do it, or that he believed himself to be part of the wider movement, or perhaps that he was deluded in imagining himself to be in Al-Qa'ida.

But it seems he actually met with the real thing, and some accounts say they gave him the explosive device, such as it was. This strongly suggests that the terrorist threat is only as bad as it was before he set fire to his testicles - and possibly considerably less so. After all, an assessment before this would have had to state that there was at least the possibility of another well-organised, multiple attack, but that the track-record suggested that it was not very likely. Now we know that they had a go, and the outcome was a single attack with dodgy chemistry and bungled tactics.

This does not suggest that we need another wave of security bingeing. Arguably, the whole point of attacks like these and Al-Qa'ida international itself is to get our attention so that their local affiliates can pull off ones like these:

Regarding reports that a suicide bomber infiltrated deep into a U.S. forward operating base in Khost, Afghanistan, yesterday, killing seven CIA officers and seriously wounding six others, a former senior CIA official posted to the region writes:

"Yes, Jaluddin [sic] Haqqani spent nearly 10 years fighting to get Khost back from the Soviets. That is his area and we were stupid to think he was going to let us stay there.

In related news, why are we not surprised that the fact he was in a database of over 500,000 suspects didn't help much? Are there 500,000 terrorists? This is one of the legs of the false positive/false negative problem - the more people you include as suspects, the less dense in information that list becomes. Rather than looking for a needle in a haystack, the problem becomes more like looking for hay in a haystack. Yes, you'll find it. Plenty of it...but what do you do then? You've just pushed the problem of filtering your information down the production line one step.

As Ackerman also points out, you have to make a decision about what to take seriously, or else the suspect list will overtake the population of the earth.

The response from the British and Dutch governments appears to be a sort of half-hearted pretend version of that after the last one; a bit like the attack.

The annual document dump under the 30 year rule is usually good for a story or two, but this year's threw up something genuinely strange. Here's Thatcher's response to the Vietnamese boat people refugee crisis; some obvious Thatcher features are there - notably sanctimoniousness, vindictiveness, and the abiding right-wing tendency to mistake sarcasm for content:

When Whitelaw said his own postbag indicated a shift of opinion in favour of accepting more refugees, the prime minister said that "in her view all those who wrote letters in this sense should be invited to accept one into their homes. She thought it was quite wrong that immigrants should be given council housing whereas white citizens were not."

(Nice Willie Whitelaw, as I believe one is legally required to refer to him, is in there as well, coming up with "a kind of steeplechase designed to weed out south Asians in particular", which would seem to be a piece of deliberate official racism.)

But this bit is truly odd:

The papers, released at the National Archives today, show that her reluctance to take in any of the Vietnamese boat people led to her making a proposal to the Australian prime minister, Malcolm Fraser, that they jointly buy an Indonesian or Philippine island "not only as a staging post but as a place of settlement" for them all. This proposal was blocked by Lee Kuan Yew of Singapore, who feared it might become a "rival entrepreneurial city".

The PM was seriously suggesting creating a new crown colony, in the year of our Lord, 1979? What? This requires a significant update in our understanding of a number of things in history, especially Anglo-Australian relations and the overseas military commitments. And you can't miss the Minister Mentor himself's swift move to prevent Hong Kong 2.0 from appearing on his doorstep.

The alternate history would be fascinating. I can easily imagine the Thatcherite-dystopia version (privatised nonfunctional water supplies, tax-haven jillionaires, enraged Filipino or Indonesian guerrillas and bewildered British infantry exchanging fire), as well as the Thatcher-fan version (low low tax city state, semiconductor fabs, banks etc). And, presumably, if the project hadn't gone all Somali by 1999, it would have taken the place of Vancouver as a primary destination for Hong Kong emigrants.

By the early 80s there wouldn't have been that much room for another semiconductor fabbing Asian city-state economy. Actually, I suspect the most likely development model would have been something like the Sharjah Airport Free Zone, and I've even got a candidate island or two - this one isn't Indonesian or Filipino, but it is an old Royal Navy base with a deepwater anchorage and an airport, that's since become a tax-haven. The runway is only 7,500 feet long, but there's room to expand at one end - although not at the other, because there's a golf course in the way and it's these people's culture, like.

Perhaps Paul Staines would have become mayor?

The actual documents are available here, although you'll have to go through a silly rigmarole of using the National Archives webstore to pay them £0.00 and having them e-mail you a download link. And they are 30MB image files tucked into PDFs. (Usability FAIL.)