Verification is often considered the wicked step-child of design. Most designers want to design because to design is to create. There is not an ounce of creative work that goes into verification. Designers tell verification engineers what tests to write and the verification engineers job is simply to code these tests with little or no input on their part on the design of the tests. It’s not surprising, then, that most engineers want to be designers.
However, this view of verification engineering misses the point. In fact, looked at from a non-engineering point of view, verification can actually be viewed as the more critical task and, more importantly to the verification engineer, the more prestigious task. It all comes down to how one defines the job ‘engineering’ as opposed to, say, ’scientist’, or ‘artist’. From a societal point-of-view, each one of these jobs is equally important. In fact, it is often the case that scientists look down their noses at engineers because engineers simply use knowledge that scientists discover. Engineers point out that most of today’s technological marvels are created by engineers, not scientists. Artists look down on both since neither of them creates anything beautiful.
The relationship between design and verification is analogous to the relationship between engineers and scientists. Thus, it is understandable that designers can look down on verification and, at the same time, verification can rightly hold a position of respect equal or higher than design.
The task of engineering can be viewed as a combination of art and science. Science is defined as the discovery of knowledge. This is usually taken to mean some fundamental knowledge about the universe, but can, in a broader sense, apply to any knowledge. For example, suppose no one has ever designed an 10G Ethernet controller in 65nm technology and you were tasked with doing exactly that. Clearly, it would be necessary to first gain some knowledge of how to do this. Since no one has done this before, you essentially would be performing the role of a scientist while gaining this knowledge. In fact, all through the design process, most of the work will be taken discovering knowledge about how to design such a product. Generalizing broadly, we can say that 99% of engineering is science.
The primary tool at the scientist’s disposal is the scientific method: hypothesize, perform experiments, conclude results. Usually, hypothesizing and concluding require virtually no time, making experimentation the most time consuming part. The most time consuming part of this effort is designing the experiment. Designing is engineering. Consider the Apollo moon program. The actual science conducted in this project was a few hours gathering moon rocks. Engineering the experimental apparatus, which is to say, the moon rocket required thousands of man-years of effort. Thus, again generalizing broadly, we can say that 99% of science is engineering.
Art is the creation of beautiful objects, engineering is the creation of useful objects. Although the output of these acts is very different, the actual process has many similarities. Most people understand that engineering useful objects is a very rule-bound process. The creation of art is also a very rule-bound process. In painting, beauty comes about because of a detailed understanding of color, composition, materials, and a number of other factors. It is possible for you or I to create art without understanding these rules, but the probability of creating something beautiful is as remote as if one randomly soldered together some resistors, capacitors, and transistors and it ended up being something useful. Music, in particular, has intricate rules that often leave little choice in what notes to choose during composition. It is often thought that genius is the ability to break rules and find new and better rules. In fact, the opposite is true. Whether it be Mozart or Einstein, geniuses do sublime work by strictly following the rules. In fact, it is their deeper understanding of the rules that allows them to see better what can be done.
We can define engineering as combining the creativity of art and the discovery of knowledge of science. Design is the artistic side, but which still requires strict discipline in following rules, which are given by specifications and other design rules. At the same time, design requires figuring out how to design something which has never been designed before. There clearly is some up front scientific discovery needed.
Verification is like science. A verification engineer is given a design, about which very little is known. In particular, it is not known whether there are any bugs in the design or what they might be. The goal of the verification process is to discover any bugs. This, more than anything, is science. Thus, if design brings out the artist in an engineer, verification brings out the scientist. Both are noble.
The training of an engineer is highly disciplined and is done with a heavy dose of the scientific method. Why, then, do engineers disdain verification so much? Much of the blame can be laid on the need for instant gratification. At the very beginning, both verification and design require some amount of discovery of knowledge in the form of studying the specification. After this, the first step of design is purely creative, the fun part of design. The first step in verification is pure drudgery, setting up test benches, scripts, and writing endless tests. After the creative part of the design is done, the rest is pure drudgery. The fun part of verification occurs after all the drudgery is complete, and that is when the verification engineer starts discovering bugs in the design.
There is no greater joy than discovering a bug that reveals some fundamental mis-understanding about the design. This new knowledge can have a dramatic effect on the design teams perception of the design. This effect is not unlike that of a scientist discovering something that contradicts some well-established principle of the natural world. The most famous example of this is probably Einstein’s Theory of Relativity which changed how we view fundamental laws of physics. This analogy is more instructive than appears. Newtonian mechanics work most of the time. Relativity corrects those cases in which it fails. But, Relativity does not simply tack additional refinements on to Newtonian mechanics. It provides a different point of view and shows how Newtonian mechanics are a special case of this new point of view. Although many bugs found during verification are simple refinements of something that is correct, bugs are found which show the true principles of how a design works and that the initial design was simply an instance of these principles that happened to work most of the time. It is these bugs that provide the real satisfaction to verification engineers.
). I will try to frame this in a way that even managers can understand, which, hopefully should help lead to better decision making with respect to planning the amount of effort that needs to go into using complex algorithms in design.
