Productivity Measurement - Is Your Program Complete?
by Steve Neuendorf (Originally published in 1994)
You got the function point count; you got the hours measured; you got some productivity metrics;--is that all there is? The real question is: did you get all of the productivity measure or did you only get the half of the productivity measure that looks like output over inputs or units of work per unit of product; the familiar Function Points per Work Month or Work Hours per Installed Function Point?
"Software’s Chronic Crisis" (Gibbs, W. W.; Scientific American; Sept., 1994; p86), points to the need for applying industrial engineering discipline to software engineering, if status as a profession (v. craft or art) is expected. American National Standard; Industrial Engineering Terminology; Work Measurement and Methods; ANSI Z94.12 - 1972 defines PRODUCTIVITY as: 1) the ratio of output to total inputs 2) the ratio of actual production to standard production . . ..
As some background, one of Industrial Engineering’s main engineering products is the Engineered Standard. Paraphrasing to software engineering terms, an engineered standard is an expected measurement (or metric) value for a software engineering task or process. Engineered standards are products of rigorous engineering and analytical techniques. Attribute analysis, attribution of measurement variation to discernible differences in common characteristics of varied tasks or projects, is analogous to software engineering engineered standards development.
This article will show you that the key to a successful process improvement program is to use both definitions of productivity.
Consider your measurement program as a wall, made of building blocks.
The foundation has two blocks to represent what is measured: 1) activities, or things that consume resources such as tasks or projects; and 2) products or those things that are the tangible results of the consumption of resources.
On this foundation are the measures that quantitatively describe products and activities, and there are four blocks: 1) size measures; 2) quality measures; 3) resource use and timing measures; and 4) attribute measures. The best example of a size measure is Function Points. Size measures tell how big or how much, and apply to both activities and products. Quality measures include defects, or other indicators of product goodness. Resource use measures are hours and dollars, and include the timing of when the hours and dollars are consumed by an activity. Attribute measures, like size measures, apply both to products and activities. Attributes are those characteristics of an activity or product that influence the value of a metric. (See the sidebar for a more detailed discussion of attribute measures)
Metrics. comprise the next row of blocks. There are three categories of metrics corresponding to the generalized better, faster, cheaper goals of a metrics program. Quality tells us how good our products or services are as a basis for determination of better. The timing components of resource use will measure how long it takes to provide products and services as a gauge for faster. Productivity metrics (definition 1 from above) is how much per hour or per unit cost, and gauges cheaper.
Far too many metrics programs go no higher, giving a good sense of what is happening but no real insight into why it is happening or how to dramatically improve performance. You may also notice that attributes do not play a part in the building of the metrics.
The next course is the key to the success of an improvement program and it has only one block: Performance Standards. Performance standard development is analyzing metric variation against attribute data determines the effect of each attribute on the metric result. In many ways this is as complicated as it sounds and requires different skills and tools than are required at any other point in the measurement process. Fortunately, there is plenty of qualified support available.
Engineered performance standards make possible the analysis of variation represented by the two blocks in the top row: 1) common cause variation analysis; and 2) special cause variation analysis. (Sound familiar?) Common cause looks at expected performance for any activity based on its attributes. Special cause analysis looks at deviations in an activity’s actual performance compared with expected performance (our second definition of productivity). Out-of-tolerance deviations (analogous to out-of-control on a Statistical Process Control control chart) indicate the need for investigation. Improvement activities focus on the identification of best practices indicated by the engineered performance standards for attributes. Experiments, or activities that have attribute(s) not included in measured activities, evaluate potential improvement innovations.
In summary and conclusion, a complete measurement program lets us follow our own advice to focus on the process. In repetitive activities, the steps are always the same from cycle to cycle, so performance follows a distribution function, and task variation is analyzed as a measurement of the process. In custom activities, only the attributes for performing the activities remain common from task to task. Attribute influences on performance follow some distribution function, and the variation is analyzed as a measurement of the process. Since software engineering in a custom process, focus on the process and attribute analysis are one in the same. The key to this focus on the process is the engineered performance standard for attributes and groups of attributes.
Attributes
Attributes are those characteristics of activities or products that affect the results of a metric. Attribute Analysis is the identification of attributes, quantification of attribute levels of influence, and analysis of the variation in metrics for those activities to quantify the metric effect of the differences in attribute levels of influence.
Attributes can be divided into four main categories: Site, System, Project, and Product, each with four sub-categories: People, Tools, Environment, and Techniques.
A good analogy of an attribute and its measurement is any of the GSC in the Function Point model. GSCs affect the metric Function Point Count, and according to the calibration of the model, the effect of each GSC is to make the Function Point Count as much as 5% larger depending on its rating. (Do not define attributes that are the same as the GSC for any metrics using Function Points, since the effect is already normalized out in the Function Point count.)
Figure 1
Metrics Program Building Blocks
Foundation: What is measured.
Measures: Data about Activities and Products.
Metrics: Information about Activities and Products.
Engineered Standards: Information about Attributes and their influence on metrics.
Analysis of Variation: Information about performance and processes.