Costing projects

I'm very aware that I should put keywords in my blog. I'm also aware that I'm not attempting to monetise my blog. Anyone who reads it regularly, leave me a comment and I will be most grateful. Anyone who doesn't read it regularly: you're missing out.

I was comparing the fantasy of Agile management to the joys of converting a house.

Firstly, no matter how tightly you plan your project, it won't work. The tyres get punctures, the render doesn't go off because it's cold, the plaster sets too quickly because it's warm and so on and so on. One of the illusions that computers peddle is that values are fixed and unchangeable. Over years of experience one discovers that an apple this year is not the same as an apple last year: it may be smaller, sweeter, less common and so on and so forth. In the same way, programmers get ill, libraries get updated, life turns out to be a little difficult. It's fine if you're building on a greenfield site, but in the real world you may not be.

So we are dealing a non-representative series of events. Estimates are based on normal - that frictionless set of billiard balls or that perfectly rational economic creature... Life is imperfect, unbalanced, frustrating, and driven by chance. People on the top want to believe that it is driven by skill. They wish to confirm that it is their qualities that have given them their privilege, and others' failures that have denied it. When talking about becoming a premier league footballer, people are willing to admit that some people are born with more talent than others. You rarely hear the fact that given a couple of X chromosomes, you'll never make it. One sperm over another. Your chances ruled out, just like that.

Given the innate unpredictability of life, you need a reserve fund to cover the likely overruns in time and cost and temper whenever you're planning a project. You also need a contingency plan.
This is one of the things I find most interesting. What do you do when a project has overrun? At what point are you prepared to say "pull the plug/cut the costs"?

It is my belief that life is much simpler if you have a hard deadline, chosen in advance, such that when you go over it, the project is pulled automatically. I would be curious to see how many things would get sacrificed if you knew that at fifty per cent above the original estimate, or at twenty per cent above the original time, consequences dropped in.

When I was in a partnersip, I rekember bidding for a contract. We honestly told the buyers "It's not possible to do y in the time, but we can do x". We didn't get the contract. We were correct, of course, but people want to hear that they can do y in time t. Anyone who says "No" will lose out to anyone who says "yes". You therefore need heavy penalties for misleading on contracts.

There is less temptation to gain a contract bby being over-optimistic if you know there is an automatic penalty for getting it wrong. But how may customers prefer illusions to truth?

Project management and priorities

As detailed before, I have the hardback book (it's green - anyone else remember the Porridge quote). It has pages and pages of lists, all neatly sorted into four sections, interspersed with scribbles, diagrams of flues, back of the list calculations and so forth.

Approximately once a day we have the discussion.

The discussion consists of:

1. What can we cross off yesterday's/this morning's to do list?
2. Oh, what did you do then?
3. Oh

Long pause

4. Well, I'll add them to the list and cross them off.
5. Where are they in the list of prioritites?
6. OK, these are my priorities
7. What do we absolutely have to get done before we move in.
8. OK, we can survive without a shower.

Heated drawing up of two lists: one of things that must be done and one of things that are nice to have
We end up with three things crossing the line (insulation, decoration and chimney cowls if you must know).

9. So when you say do floor, what exactly does that involve?

Calmer and much longer list of all the components of the "things that must be done" list. During the discussion, we discover other things that must be added to said list.

What user interface design point am I demonstrating (or any other point)? I would say that it's a technique that is a refinement of Bill Buxton's Sketching User experience.

It is only when you have your ideas out there in some form that you can discuss them. I cannot stress this enough.

What I have in my head is non-negotiable because nobody knows about it.
What I say is somewhat negotiable, but it is often not clear what I mean.
What I sketch or model or put on paper is utterly negotiable because there is something to negotiate over.

Here is a mock-up of the stairs to the basement. It's a starting-point.
They're not on the priority list though.

Houses and the Harvard method

Do you know about the Harvard method?  I'm not talking citations or negotiation; I'm talking making lists. I was told it by a friend of mine and it's the best thing ever. (That, obviously is a wild exaggeration. I can think of many things that are much better, including champagne, my daughter's GCSE results, the smell of jasmine on a summer evening, etc etc, but in the breathlessly excited tone that is required for this sort of blogging, I will let the statement stand.)

Step 1: You take a piece of paper, preferably A4 or letter but it can be any size

Step 2: You divide it into three or four sections using a pen, pencil or other writing implement of your choice

Step 3: You write your list

The amazing and wonderful thing is that the process of thinking "which section shall I add this item to" causes you to sort the list items on the fly, and helps you to prioritise.

Because that point is, after all, the key struggle in project planning. How do you set your priorities? Do set priorities, you need to know what your goals are. And if you haven't recognised your goals, then they change and flip around and mess up the priorities. I've been reading a book by Gavin Essler, "Lessons from the Top" (no, I am not going to cite that correctly in italic and publisher and blah, I'll give you a link, isn't that enough? And it's not to amazon because of their behaviour on UK tax - correct but amoral.)

Sorry, got slightly distracted their. Essler quotes Alistair Campbell as saying that the key questions are: objective, strategy, tactics. And if you lose sight of the objective and get distracted by the cleverness of the tactics, you will fail. The tactics must always be subordinate to the strategy which must always be subordinate to the objective.

This is true for all projects.

An example of how it goes wrong (substituting stategy for objective) is to think that the point of the house conversion is to have a beautiful house. It's not. It's to have a happy family life. If you get stuck on the beautiful house bit, then you invest all your resources into the house, lose money, have enormous rows and go bankrupt, get divorced and everything goes phut. Having a nice place to live is a way of assisting the objective. And the method of converting the house to make it a nice place to live is merely a tactic.


So when you sit down in the morning or the evening with your piece of paper (or in our case a hardback book) and draw the lines on it, and start sorting the items in your list. Think clearly, think long-term. Some of it really doesn't matter that much, but sometimes, short-term goals will hinder that final objective.

Feature development and house conversion

I've not been blogging for a while because we have bought a fixer-upper house which needed a fair amount of work. I can't help but be struck by the horrifying similarities between house conversion and software development.

1. The estimates are always too short: people calculate the work period, and even when they're being generous, they forget to allow for being pulled off onto another more important (paying) job, set up costs, or the time spent searching for the library routine/thermostat which will do the job you want rather than all the others which nearly do the thing you want.
2. The code/wiring/plumbing that you have to interface with is always spaghetti and it takes much longer than it should do.
3. The code/drainage/etc that you have to interface with has not been documented and you will need to follow it through twisty strange paths and try and work out where the spurs go.
4. Even if it used to be fine, old code/wiring/plumbing does not match with current building standards/operating systems and has to be re-done.
5. Good programmers/builders etc cannot resist the temptation to do something well rather than well enough. This can take a long time.
6. Ambitious heads of engineering/house owners will suddenly have a brilliant idea about how something could be done better and require the whole thing to be re-designed half way through. They may well be correct but it adds a great deal to the time and cost.
7. It is disheartening to see no apparent progress.
8. There is a struggle between designing a usable interface and a functional system. Sometimes the people who have to do the extra work to make something easy and pleasurable to use rather than the way they'd first thought of it can be resistant to extra work.

So, given the fact that all the engineers/builders involved are intelligent craftsmen, how can the process be less painful?

I decided to apply some of the principles of Agile project management to the building project.

In Agile management, there is this fantasy (and it is a fantasy in many cases) that you can work on a discrete feature and get it through design and test in a solid blast, close it off, and then go onto the next thing. There are various buzz words around it, such as the scrum, the daily stand-up meeting when everyone talks through how they've got on and what needs to be done.

It's not a place for novices.

Solidified thought and an Inkscape review

Life has been busy lately. I've been revising the test schedules; not my absolutely favourite thing but better than a poke in the eye with a blunt fish. After the first desperate loops round the build, build broken, build buggy cycle, we have reached the stage of catch-up that we should have been in to start with, where features are being cleaned and polished, and software updates are being released every couple of days.

Apart from work, I have bought a house, and have been using Inkscape to plan the structural changes. It's a nice tool. I started using Autocad Architect, because I can get it free as a student, but it isn't merely using a sledgehammer to crack a nut; it's using a Boeing Dreamliner to commute down the road to work. Yes it's possible, and no doubt once it's all set up it would be quicker than walking, but the learning curve is so steep and time-absorbing that I'll just slip my shoes on for now.

OK, why do I like Inkscape?
It's pretty basic, but it's a proper vector graphics program, and it has all the things that I like using, in terms of converting objects to paths, grouping, layering etc. It lays things out properly with co-ordinates, and gives you accurate measurements.

There are some things that I'm pissy about - such as the way it seems to scale things proportionately when you don't want it to, and I look forward to discovering about using 3-D stuff.

You can import bitmaps, and export bitmaps, so I've done some beautiful scale drawings of walls and windows so I can do accurate sketches of the planned mural.

All right, I admit it. The whole Inkscape exercise was a procrastination. It enabled me to spend a whole evening doing scale drawings. Possibly even two, and I haven't done a single sketch on them. I have, however, borrowed a nice large book of tree photographs from the library. Get me!

Naming names at the Cheltenham Science Festival

I didn't get the grant to attend the Cheltenham Science Festival (though I was runner-up, thanks UCL). As a result, everything I currently write is entirely unbiased and paid for by myself.

I attended several events today:

1. The Web and Us

2. Conversation without Words

3. Stand-up maths

It should have been four events, because there was a follow-up to Conversation Without Words of a tango lesson (as an exemplar of conversation without words) but we were fortunate enough to find a buyer for a tickets.

Right, report back. My name is now on this blog, so I have to be willing to stand by what I say.

The Science of the Web didn't say much that I didn't know, but it reframed my knowledge. This is never a bad thing. My quote of the week (and possibly the month) was provided by Aleks Krotoski quoting Melvin's Kranzberg's first law of technology "Technology is neither good nor bad; nor is it neutral.".This was in response to a question of mine (self-aggrandisement is not good, nor is it bad, nor is it neutral) about the fact that the web was designed and created by a specific sub-section of the human population and how does the virtual society it supports mirror that social group. The panel for this discussion (the aforementioned Aleks, Uta Frith, Nigel Shadbolt, Bill Thompson and facilitated by Wendy Hall) was extremely (a) intelligent and (b) optimistic. Having read "The Social Life of Information" I'm not sure that I'm as optimistic as they were. I would like to name-check Bonnie Nardi, but I have a horrible feeling that many of my concerns are based on family experience, and after listening to Aleks Krotoski talking about the identifiable behaviours of your online anonymous persona, I don't want to admit to anything. Any where.

The sceond talk I went to was on conversation without words. This talk embarrassed me so much that I emailed one of the headlined participants with my criticisms. It was so bad that I'm not even going to say who that was - you can look it up if you really want to know.

The final session of the night was Matt Parker's Stand-Up Maths. I've seen him before, and he was just as entertaining this time as he was the previous times (which for sad, geeky people like me, quite a lot). My only regret is that I'm not quite sure if I learnt anything from the experience, and let's face it, people like me only fully value experience if they can come back and wow you with what they've learnt. anyway, it's late, and having not gone to Botany of Gin lecture, I have been doing my best to make up the shortfall here. I think that I now need to turn my main (and tastebuds) to Bombay sapphire and leave the blog for another day.

The value of big data versus confirmaton bias

The first thing is a massive Yaaayyyy! And possibly Hurrah, hurray, hurrah. We have released. Doctor's surgeries all across the land are getting their beautifully designed boxes, or their download invitations, and installing OUR software on their systems. Ian and Mr Grumpy are busy de-bugging as fast as they can, ready to release an update straight away, but marketing and sales are hap hap happy. Gavin and David are cheerful for the first time in weeks.

Obviously one of the things that sells the software is the way you can analyse your data. So you can see hours of work and rotas in pretty little patterns. In theory it will even calculate the nurses and doctors rotas for you. You can connect it to your appointments system and see who works hardest and spends the most of prescriptions and so on and so forth et cetera et cetera et cetera. GandD's cunning plan is to allow people to register their data (anonymously) and see how it compares with other people's data

The question is then what do you do with the data. As I've mentioned earlier, quite frequently, there are people who change their mind because of information, and people who don't. This may be for two reasons. One is that well-known problem (or achievement) of human psychology; that we prefer information that backs up the position that we hold. We look for facts that support our viewpoint, rather than facts that will disprove it. This is why the null hypothesis is such a fabulous idea and is more common in myth than in reality.

The other problem is that the data may be useless. Firstly, it may be data rather than information. Information is data which has been judged, and it takes a human to judge (or, of course, an omniscient deity). Given humanity's penchant for partial information and prejudice, we would often prefer to pass our judgement elsewhere. Secondly, it may be useless if one can take no action based upon it. What is the use of information that cannot be used?

So maybe, it's best to hide data. Leave it in its data buckets in the garage, unmined and ignored. After all, all that unused data will only clutter up the minds and computers of the people who have to look at it. It will distract them from the decisions that they wish to take.

Perhaps I should introduce the concept of small data. Instead of "just-in-time" information, "just enough" information. Let the rest of it sit around if people want to go fossicking through it because they have an idea, but otherwise, don't encourage them to mess about with it. Concentrate on the things they want to do, such as where to go for lunch and how many times you can read the same detective story with a different title.