Looking into contemporary definitions of a software defect you can find a load of nonsensical definitions. At least in my opinion they are. Here is my definition: »A software defect is the Absence of a required function, the incomplete implementation of a required function, the erroneous implementation thereof or the unwanted presence of a function/behavior.«
There is a better definition of a software defect and I read it years ago, but searching for it today yielded lots of nonsense like this »A more general way of thinking about defects is “anything that threatens the value of the software to some person who matters.” « And I call it nonsense to because I consider my self a polite and considerate person. One thing is certain: If you lack the attitude to nail the problem, you’ll not solve it. It’s an attitude-problem.
I just was at the dentist and guess what, they had a software problem. It was one of these typical weird behaviors that software developers would have a hard time to fix, because they should have addressed this issue before they started writing the first line of code. I opened my mouth, everything is fine. She recommended to get two overlays for my teeth we talked a little and I left her office thinking about that issue another time – I believe, I thought about it a million times. Looking at contemporary job advertisements for software quality assurance engineers, there always is very clear focus on testing and agile and stuff. It’s the usual buzzwords, non of which is missing and I shake my head and go my way.
Next thought was that by testing you’ll only find the defects you are looking for.
Next thought was: When did I encounter this erroneous trend first? That one I can remember, I thought. It was in 2000, when I hired at ESO and I remember the job interview very well, it was a kind a unique. ESO is a scientific organization and the job interview was in the lecture hall with a group of four scientists and it’s not only that they interrogated me as scientists tend to, I also came up with a few questions, for instance about the documentation and I remember their answer as if it was yesterday. “Of course, we have documentation, but we don’t overdo it.” I was too naive and I wanted the job in a scientific organization, so I was happy when they hired me. I was in charge of fixing bugs for the P2PP software. I found no documentation at all. Not a single page! Instead I was given a list with hundreds of software defects. The software was written in Java 1.1.8 and the three software engineers, who did it, left the organization. They were all freelancers. This software was a funny piece.
I continued to ask for documentation and finally I got the answer that I should look into the book “Design patterns”. This was the documentation and the software developers were tightly following that book.
When you think »bullshit!« at this moment, you are exactly right! So they already had a way of quality assurance and that was unit tests. So for every bug, fixed, you developed a unit test to detect a potential regression – the re-introduction of that old defect in a later version of the software.
So I sat down and tried to work to untangle the weirdest code I’ve ever seen. These software developers re-invented the wheel over and over and for already complicated requirements they implemented exceedingly complicated solutions. The first phase of my task was archeological. I had to dig to find out where the problem was and they usually were buried deeeeep in that code, somewhere in that homegrown object-manager and they were inherited over a plenitude of levels . The deeper I dug, the fewer the comments were in that code. You could see how stressed out these software developers must have been and how they finally wiped away their sweat when they believed that it finally worked and forgot to write a single line of comment or to correct the no longer valid comments that were written there at the beginning. There were lots of comments that only told the absolute obvious, like // loop until the end of the list and similar nonsense. My ass, I can see this, nobody needs this comment there!
A few weeks later and after continued complaints I finally received the documentation. It was five pages and they said that this piece was a manager/singleton pattern and that one was a singleton/factory…. a facade….blahblahblah five pages and not even full pages.
So I wrote a template in TEX and for every bug I fixed, I wrote a report with vi, addressing the crucial points:
- Title and Description of the defect
- Nature of the problem a more precise description of the expected behavior and what really happened.
- Where the defect was located
- How this particular defect functioned in detail and what development approach and technique caused it
- How I fixed it
After a few weeks I gradually learned that my manager hated me for that. He only wanted the “bugs” to be fixed and reporting what I actually did was very undesirable for him.
Walking to back from my dentist I discovered that this was not the first case of asininity in software development. I encountered it first when in 1999 a colleague tried to convince me that they had so many libraries in C++, things we just don’t have in C and I thought, “Well, you just don’t know the C-libraries. It’s all there!”.
Many software developers – I don’t call them software engineers, because software engineers know their libraries – shifted to C++ because they believed they had the better libraries. They guessed that C++ would be beneficial for their careers and so they started compiling their C-code with a C++ compiler, replaced printf() with cout and Voila! It’s C++, right? (see: Gee! Yeah! I am a C++-programmer :-/ ) while their code still, even today, has nothing to do with object orientation.
It didn’t get better. I came in touch with the CMM and PMBOK in 1994 and we trained and worked hard to implement it. After I left Motorola SPS, I found myself in environments, where software quality assurance didn’t play a role and whenever I suggested to please at least look at the one or the other method, I yielded pity and laughter.
In 2000 I lost my job as a manager for a software development department, because I had a very good relationship with the new CIO and was totally in line with his demand to beef up quality assurance, since there was a legal requirement. So he offered me a position to build up that quality assurance and my direct superior vetoed it, because I was working on process while he only wanted “documentation”. I ignored his veto and continued to work on that topic and he fired me as long as he could. This was the state of quality assurance in a major German telecommunications corporation in 2000.
From there I went to ESO and from there I became a freelancer. And it never became better. In 2005 I worked as a consultant for Infineon to help them implement CMMi, but they had their own concepts of implementing it and they only paid me for giving them the advise they wanted to hear. So they tailored the original, which actually would be a good beginning, but they produced tons of paper and they ended up with almost three thick folders full with templates and the templates were many pages thick, so that the engineers only had to fill in the gaps. The problem with this was that you couldn’t read this any more. You ended up with piles of documents, all looking the same and which didn’t differ except for the few sentences entered by the engineers. The engineers protested, I protested, but it was implemented that way, no matter what. And I have seen this happening over and over again.
I can’t remember how often I recommended to use the Team Software Process (TSP) instead and for very small organizations to uses the Personal Software Process (PSP) all of which, you could obtain ready tailored thin documents from the Software Engineering Institute that just waited to be grabbed and followed and start from there. Then implement a process oriented quality assurance, applying metrics nailing the root causes of defects and tackle the processes to remove these root causes and from there implement implement CMMi compliant processes. Man, this would have saved so much energy and so many trees, but they never listened.
I’m still convinced that one can implement the CMMi for a complete organization and describe it with a document of less than 300 pages and I am firmly convinced that a template should only specify the general structure and the headlines. You will not fix it if an engineer is dyslectic. No matter how enormous your template will be, at the bottom line it will only contain nonsense. And when you then just check off the document you absolutely missed the intrinsic intention of quality assurance.
I’ve seen requirements documents with more than 10,000 requirements and nobody in the Change Control Board (CCB) knew what they were talking about when they were kneading their huge lists. Most requirements were just a few keywords such as “frame rate 30” and no further specification. So it was no wonder that the one or the other feature never was implemented correctly. Instead of focusing on how to write a requirement correctly, a use case and a user story, they focused on the tool and finally on the tool chain, guided by the believe that a ClearCase and a ClearQuest and CaliberRM or RequisitePro, HP Quality Center…… the bulkier, the better, would fix things, but these bulky software tools rather made the lives of the software engineers harder and they leaned towards only using a tiny fraction of the functions provided by these and other tools, purchased for hundreds of thousands of dollars for a corporate license.
It was assumed that a push towards object orientation would fix a lot of software problems since it was insinuated that the engineers predominantly following imperative paradigm were unable to implement increasingly complex requirements. Look where we ended up today. Nothing became better. Instead much has become worse, because we never addressed the root causes of defects and hence we never went through the hard efforts of the wide and dry plains of engineering.
But with tools and testing we’ll never cure the problem. In the best case, we will detect the defect after the fact.
So how can we fix this? << this will become a link to the next Blog.