Software Development Paradigm Trap main page

Feedback: Software Development Paradigm Trap

Greg Nelson - 8/18/2006 - responding to Steve Fairhead

Steve,

I enjoyed your response to Mark Bereit's proposal, and I very much agree with your conclusion that we don't spend enough effort on design. (I thing this is borne out by a lot of research efforts such as Watts Humphrey's "personal software process" work.) I am curious, though, how (barring typos) one can always arrive at correct designs without test and iterative implementation.

It seems to me that the ideal of "correct design" would conquer the old "garbage in, garbage out" rule. But I believe the only way this is possible is to be able to unerringly identify at least one of "garbage in" or "garbage out" and flag it. Am I mistaken, or do you mean something more constrained when you refer to "correct design?"

Certainly, we can sometimes do this, in applications where the data are "well structured." I can make a practise of checking for division by zero and the square root of a negative number. Things like semaphore signalling and priorities and state machines can be analysed and proven either (A) correct, (B) incorrect, or (C) indeterminate. Too many software writers allow indeterminate cases to be treated as "correct because I tested it once and it didn't break." Then there are cases like financial and database applications: the books either balance or they don't; a year either has enough digits to work in a different century than it was written it, or it doesn't.

There are other "poorly structured" cases where I can't fathom how to establish similar guarantees. For instance: we make measurement systems that need an automated calibration routine. We have a pile of spectral data and somehow need to determine which measured datapoints correspond to which calibration datapoints. For every algorithm I have considered or implemented over the years, I have either seen or been able to imagine perverse cases (some due to different physical conditions, some due to user error) which will throw the algorithm off. (Then, if the user uses the system with a detector we don't sell, I can't even begin to predict what the data will look like.)

I'm certainly interested in suggestions or references for how to ensure correctness where continuous data needs to be discretized for analysis, but I don't think there are many of these. The multiple-algorithm-voting approach probably qualifies, if resource-intensive.

The other challenge of proving things correct involves our tools. Even if I could prove absolutely the correctness of a set of application code, if I am running it on an RTOS that has bugs, it may not behave according to design. I think this is a major contributor to the "starting from scratch" mentality that is so reviled and yet so common -- it is easier to debug an RTOS that you know intimately because you wrote it yourself, than one provided by a vendor, especially one without open source code.

Sincerely,
Greg Nelson
Engineering Manager
PGT Instruments, Inc.

Steve Fairhead - 8/18/2006

Hi Greg,

Thanks for your thought-provoking email.

Lately I've been refreshing my memory and researching methodologies and approaches; I'd forgotten about PSP. I've read a bit about it; so far, I agree with much of it. Will read more.

I am curious, though, how (barring typos) one can always arrive at correct designs without test and iterative implementation.

My answer is complex, but boils down to this: the only way to write code with no errors is to design it to have no errors. Dijkstra said it much better: "Program testing can be used to show the presence of bugs, but never to show their absence."

I do believe in testing, but I don't believe in debugging. I often see debugging represented as a graph of bug content vs time, showing a curve that asymptotically approaches zero - but never quite gets there. This seems a strange way to do engineering. Furthermore, time planning is impossible - that curve is likely to be truncated by time constraints long before "an acceptable level of bugginess" is reached. Finally, I don't believe that there is such a thing as "an acceptable level of bugginess." We should be designing for zero bugs.

As to how to do this, it's a combination of things. Mostly it's complexity management: breaking things down into small, manageable, testable, and comprehensible chunks, communicating only through well-defined interfaces, ideally synchronously. These ideas are well-established in hardware design; perhaps the fact that I'm a hardware designer turned software engineer is why I care so much about this subject. I have much more to say about this; I'm in danger of writing a book ;) .

Do you mean in terms of the specification, or in terms of the data processing? I fear I'm not following you.

Certainly, we can sometimes do this, in applications where the data are well structured. I can make a practise of checking for division by zero and the square root of a negative number. Things like semaphore signalling and priorities and state machines can be analysed and proven either (A) correct, (B) incorrect, or (C) indeterminate.

Wiser men than I have expressed doubts about the viability of formal proofs of software. However, I strongly believe that software should be (and indeed can be) so clearly written that one can read it and be sure it'll work as intended. I take the comment "but it's trivially simple!" as a compliment.

Too many software writers allow indeterminate cases to be treated as "correct because I tested it once and it didn't break." Then there are cases like financial and database applications: the books either balance or they don't; a year either has enough digits to work in a different century than it was written it, or it doesn't.

Indeed.

In such cases, would a human be able to infer the correct algorithm to use? If not, then the problem may well be unsolvable - if the data isn't there, no algorithm will synthesize it. (I'm sure you know this.) However, a good algorithm should at least be able to flag such cases, as you suggest.

I had a similarly intractable problem some years ago. In the end, we brainstormed and eventually came up with a system whose eventual output was a confidence factor. We trained the system, tweaking the weightings, until we could define a confidence threshold in which we had, er, confidence. I hesitate to call it an expert system, but certainly it was along those lines.

See above - that's essentially what we did. Re resources: do you mean in terms of code space? If so, surely that's cheap these days? Or am I misunderstanding your point?

Heh - another soapbox of mine ;) . With few exceptions, I'm not a fan of RTOSs in general. Possibly I'm biased; my experience of them has been salutory. Too often some technical manager has sought to save time by introducing unnecessary levels of complexity. Also, I personally prefer cooperative multitasking, since this tends to reduce synchronism issues.

Must dash. Apologies if this is a bit off-the-cuff; I wanted to respond quickly rather than wait, think too much, and wind up lumbering you with War and Peace ;) .

Best,
Steve