I use the term prediction tool broadly to refer to any implementation of a predictive model or prediction technique, algorithm, or heuristic. I include in the definition any facilities that perform essential input functions such as data identification, extraction, and sanitization, as well as essential output functions such as reporting, summarization and presentation.

Types of Tools

Use of prediction tools is very rare, although we know of several successful examples. I’ll contrast two typical manifestations of such tools:

• A: stand-alone, one-of-a-kind, intermittently applied facilities (usually collection of scripts and small applications) often intended for use by experts who have developed or commissioned them, and
• B: stable, deployed components, applications, or services that are part of an existing software development environment and meant for regular use in software teams.

Manifestations that are a mixture or fall somewhere in between exist, but they won’t be useful for making my point.

Tools of type A are most common. Almost all of the noteworthy examples I know of fall under this category. They are relatively cheap to develop, but each application requires expert hands. They are meant to be used by a small minority of very select people.  That alone pretty much explains their home base of large organizations with dedicated metrics, research, or process improvement departments. Indeed, at a session of the International Software Engineering Research Network‘s annual meeting this year (which happened to collocate with PROMISE), we have seen some pretty nifty examples from ABB and Avaya Labs. Type-A tools are thus fine for a large organization where the cost of the required expertise can be amortized over big, corporation-wide initiatives, however they are too resource-intensive for broader adoption.

The alternative to this “do your magic and throw the results over the bench” approach is type-B tools, in which higher up-front development costs by those handful of experts can be amortized across many organizations over more frequent use by multiple roles in software teams themselves. The scope of each use may however be small. This attractive alternative doesn’t necessarily imply that no fine-tuning, hand-holding, or maintenance will ever be needed, but it does imply making information more available and friendly to software teams and decision makers, thus giving them more control over their projects and achieving better visibility.

So let’s limit the scope to type-B tools, say because I’d like predictive modelling to be more readily available to mere mortals. What factors should we look at next to increase the chances of adoption? I’ve organized these factors in two dimensions:

• factors specific to the goals of prediction, and
• factors that encourage tool adoption in general.

Specific Adoption Factors

Predication does not happen in a void. We do it for a purpose. Among factors specific to the goals of prediction are performance traceability, easy calibration, and ability to support decision making.

If we are making predictions about cost, schedule, quality, productivity, or profitability, at one point we will probably know the actuals of the predicted quantities.  Performance traceabiltiy refers to the capability to track and record those actuals when they materialize, and then compare them with the original predictions. Such capability completes the tool’s feedback loop and gives it  the much needed visibility for continued use. If the tool is performant and this can be proven, the incentive to keep using the tool increases.

The next step up is calibration, well, more precisely, easy calibration. Easy calibration is giving the tool the capability to tune and re-tune the underlying predictive models when new data is available or when existing data becomes obsolete. Ideally, we would like such calibration to occur both incrementally and automatically. Hence the adjective “easy”. Performance traceability is a prerequisite of easy calibration.

The final step up is the ability to support decision making. Think of decision support ability  as “going the last mile,” in that we’d like the prediction tools, or the outputs thereof,  to be both purposeful and actionable. A tool is purposeful if it supports concrete goals for specific goals in a software team. A tool is actionable  if helps the user to take a growth, improvement, corrective, or preventive action. Here is an example of purposefulness, expressed as a behavioural requirement, from Brian Robinson’s presentation at ISERN:

As a Product Manager, I need estimates to include the complete cost of new features including post-release maintenance so that NPV forecasts are more accurate.

Something that is both purposeful and actionable is:

As a Testing Manager, I want defect predictions to rank modules in order of pre-release defect risk, so that I can allocate testing resources optimally.

The Testing Manager is asking more than just defect density estimates. She’s asking for information and advice that will directly help her ration scarce sources. One way of improving a tool’s decision-making ability is to use a methodology such as GQM (Goal-Question-Metric) when designing the tool. Another is to provide intelligent, role-specific filters and workflows. Role-specific filters will slice and dice the information in different ways, each filter customizing dashboards, reports, charts, and advice with a single perspective in mind. A role-specific workflow will mimic a team member’s implicit decision-making or sense-making (thanks to Mika Mantyla for the term) processes.

General Adoption Factors

As for factors that facilitate tool adoption in general in the software development context, I’ll briefly invoke the relevant dimensions of an assessment method called WeightPrints that Piotr Kaminski and I developed many years ago. Piotr is a savvy tool developer, who has been making his living creating tools for Google developers. The basic idea was to help developers and researchers gauge how heavy-weight a software development tool is for a given purpose. Our grand assumption was that lighter-weight tools were easier to adopt than heavier-weight tools for the same purpose. Thus, all else being equal, a tool with a low weightprint would be preferable to a tool with a high weightprint.

The four dimensions of the WeightPrints model that are applicable in the prediction tool space are obtrusiveness, management burden, ramp-up, and isolation. The lower a tool scores in each of the dimensions, the lower its weightprint is. Obtrusiveness increases as a tool’s input gluttony (requiring lots of manual input), conspicuousness, disruption to existing workflow, and passivity increases.  Management burden increases with installation, configuration, and healing effort as well as with the tool’s disregard for standards and its total installed footprint. Ramp-up increases as learning effort and the dependence of the tool’s success on network effects (its adoption scope) increase while its virality (its ability to spread fast), synergy with other tools, and frequency of use decrease. Finally, isolation decreases with the tool’s compatibility, interoperability, extensibility, and openness (especially, in terms of the underlying data exchange formats). But enough about WeightPrints, for I can always discuss it further in a future post.

Some kinds of tools, like prediction tools have good reasons to be on the heavy-weight side of the spectrum. However, we should strive to make them lighter and lighter if we care about their adoption by end-users.

Acknowledgments: I’d like to thank the organizer Ayse Bener and the other panel members Gunther Ruhe, Mika Mantyla, Barbara Russo, and Burak Turhan.

For program and registration visit SES ’2011 page at http://www.computer.org/ses11.

This post is password protected. To view it please enter your password below:

Password:

As the common context, I used that of a mid-sized organization in the health informatics sector with new ventures in its project pipeline and with several new ideas for improving its software processes. The examples were mostly drawn from this context and organized as HBR style cases. The cases addressed project staging, incremental development, pair programming, code inspections, role of evidence, and test-driven development.

The event was a repeat of an earlier course I gave in June for for Siemens in Erlangen, Germany. But I changed the materials and format significantly to fit the new  interaction style and  in response to the feedback I have received after the Erlangen version. I divided the material into 5 modules, incorporated smaller examples, and added take-home exercises for each module. I also frequently used LiveMeeting’s poll feature to engage the audience. All in all, I think the format worked very well, surprisingly with very few technology glitches. The tech support supplied by host was very good.

The course description is available here. If you were a participant and haven’t filled out the course evaluation form, you can find it here. I have opened this post for comments: if you have any, well, what are you waiting for? Do fire.

In A Process That is Not, the November/December edition of my IEEE Software column, I tell the story ofTikiWiki.    This is an interesting and telling story for those of us who preach software process.

A related point of view is provided by James Bach in an early article he published in IEEE Software in 1995. See Enough About Process: What We Need Are Heroes. In that article, Bach defines process heroism as:

I’m fascinated by the fact that people who take initiative to solve ambiguous problems are key to the success of every software project. I call this software process heroism, based on Joseph’s Campbell’s description of the hero’s journey. Unfortunately, it’s a subject that provokes a lot of strong feelings and confusion. There seems to be a lot of fear surrounding the simple idea that people are not easily controlled, influenced, or interchanged with one another.

I think this controversy is the single biggest indicator of how immature our field is. We can’t even agree on what we are doing or what we mean, as people, in the midst of our process creations. I believe we will never be able to control software projects well, unless we give up on treating software development as a mechanical task, and start embracing it as a human system. Anyway, that’s what works for me.

Hmmm. Food for thought…

In June 2009, the Software Architecture Challenges in the 21st Century Workshop sponsored by IEEE Software, University of Southern California, and University of California Irvine attracted quite a bit of interest. Pekka Abrahamsson, Philippe Kruchten, and Grady Booch all addressed the role of software architecture and architecting in agile software development. I ended up writing an editorial based on the Kruchten and Booch talks for the September/October 2009 issue of IEEE Software.  The editorial is also featured in the August 2009 edition of Computing Now.

In that editorial I explore the common themes raised by Booch and Kruchten, and also touch upon where the two gurus differ. The common themes I identified are:

• The scope of software architecture
• Architecture-phobia
• Architecture’s omnipresence
• Architecture for self-governance
• Architecting as a continuous activity
• Just-enough architectural representation

These themes made me reflect on my former views on the topic of how agile and architecture interact. Those views were expressed in a tongue-in-check article that I had written for Agile Times back in 2003 .  The article was sarcastically titled “Let’s Scale Agile Up”. In those relatively early days of agile, I used to advocate a purist agilista approach that downplays architecture. I used to object to any attempts to scale up agile software development by augmenting it with explicit architecting. I was wrong. I realized how wrong I was in a recent project.

The project was small, but I couldn’t crack it with seat-of-the-pants, “let the design emerge” attitude. The project’s success relied on the ability of a complex core technology to solve the problem at hand. We had to nail down technical decisions at the beginning. And these decisions couldn’t be reversed easily later, even though the project was small.  My client demanded to know about the main components, how they would interface with each other and be integrated with the core technology, how they would be decoupled from each other for maintainability and extensibility,  and what general functionality they would provide. He wanted me to document the underlying decisions. In a nutshell, he demanded that I do some architecting.

I ended up spending a week and a half on these issues, constantly soliciting clarifications from my clients on the requirements, running endless spikes to resolve technical risks pertaining to the core technology, and coming up with a general design in the process. And I am glad my client insisted because this upfront activity paid off.  It turned out that seemingly subtle aspects of the requirements had an impact on how the core technology, a hairy rule engine, would work to solve my client’s problem. Interestingly, I no longer associated all this early activity, as I did back in 2003, with the “grit” between the gears, the grit that contaminates the project and brings it to a screeching halt.  That grit did not interfere with planning and running the project as a highly incremental and iterative one. Actually, it turned out to be a lubricant rather than a contaminant. I grew up, I suppose.

No, they are not superstars, or all-knowing super-powerful CEOs poised to control the world. Although one reader, and letter writer, thought that’s what I meant in my latest IEEE Software editorial titled The seven traits of superprofessionals.

The letter writer states that he has been researching the reasons of  the recent economic crisis and states that he’s found a strong indicator of what kind of management behavior the future requires. Apparently, it’s not exactly what my editorial describes (never mind that my editorial was not about management behavior at all). The letter writer attaches to the term “superprofessionalism” — which I coined to describe a set of behavioral traits that represent a sort of high-order professional conduct — the same connotation as the false, shallow stardom associated with the kind of people featured in reality TV series like Supermodel, The Apprentice, or  The Bachelor.  (As a synopsis, the traits mentioned in the editorial are: focus on individual responsibility, acute awareness, brutal honesty, heightened sense of fairness, resilience under pressure, attention to detail in perspective, and pragmatism first). He alleges that those kinds of media programs and my article represent the last dying attempts to revive an old-economy thinking in which all-knowing individuals hold and dispense all power. He adds that my article advocates individual and central decision making at the expense of group decision making, and is counter to holistic team behavior and denies the emergence of information and collective intelligence from decentralized interactions in social networks. But the best stretch is when he puts the blame for the current economic crisis on the kind of thinking my writing encourages, like, say brutal honesty (he picks on this trait in particular, stating that it’s socially unacceptable).

Oh? Interesting perspective, although it completely beats me how any of that can be inferred from my editorial. If it could, that perspective would represent a truly curious internal paradox given my links to the the agile software development community, and in light of my previous column on The role of diversity in software development.  But one can never underestimate the intellectual depth of those who can with great facility read the sub-sub-subtext between the lines. Honestly, can anyone detect any nuances in that writing,  implanted with the idea to recreate or preserve old-world power? And me at the center of the conspiracy?

Cheers

Number 10. Ignore reviewer feedback.

Number 9. Stick to clichés.

Number 8. Borrow without giving due credit.

Number 7. Don’t worry about the language.

Number 6. Ignore your work’s limitations.

Number 5. Ignore the context and prior art.

Number 4. Rely on argumentation to push an idea.

Number 3. Ignore your audience.

Number 2. Be undeservedly authoritative.

Number 1. Write in a purely descriptive style.

In this post, I’ll address the top five sins from Number 5 Number 1.  

Sin #5 is the sin of laziness, or the appearance of it. Readers want to know what encompasses an idea, a piece of research, or a narrative about an interesting experience. They want to be able to position it relative to other works that are similar. They want to know about the connections between concepts, the progression of ideas. They want to understand the context. Prior art and context provide the beginning of a story. Without the beginning, the story struggles to survive. Without prior art and context, a paper has little value.

Sin #4 is the sin of defensiveness. Relying on winding questionable arguments to push an idea makes it seem like the author’s premise doesn’t hold water, as if the weakness of the idea needs to be hidden behind a veil of twisted logic. A better alternative to using argumentation is presenting facts and existing evidence. If evidence is not strong, just admit it. No apologies are necessary.

Sin #3 is making no attempt to differentiate among the various types of audiences. If you write up your excellent research in an overly scholarly style for a professional magazine, it will bounce back. If you write up your exceptionally insightful project experience report in an overly colloquial style for a research journal, it will bounce back. Style matters to your target audience. In addition, certain elements need to be in place in different types of publications at varying degrees. A comprehensive, academic-quality literature survey may not be necessary, or even overkill,  for a magazine, but is essential for an archival periodical. Know your audience, and know what each type of publication requires to meet its audience’s expectations.

Sin #2 often accompanies Sin #4. It comes from the absolute tone that authors often use when expressing what is simply a point a view. Or, it comes from the tendency to pepper an article with unsupportable statements or embellish ideas with virtuous qualifiers and superlatives. This almost always makes reviewers recoil with disgust and fury. Avoid being unnecessarily absolute and self-assured. Humility, without being deliberately fuzzy, will work better than you think. However, it doesn’t imply that you shouldn’t take risks. Your message can still be provocative.

Perhaps the best strategy to use to get a paper rejected is to take a good idea and just write it up. And stop there. This brings us to Sin #1. A purely descriptive style is one in which you just describe something — whether it’s a concept, a process, an idea, a new technology, an experience, a study — in gory detail, definition by definition, step by step. You start from the beginning, explain all the elements in a clear, natural sequence, perhaps give a few small examples, and then you’ve done your job. You can stop. What could be wrong with this straightforward approach? Nothing. It may be perfectly OK for a white paper, a chapter of a text book, an appendix, a manual, a handbook, or a technical report. But it won’t make very interesting reading and it certainly will not buy you any brownie points with the reviewers. Try it!

Releated resources:

• Tips for Software Authors
• How to Write a Good Technical Article

My IEEE Software column on the topic is also featured (Cloud Computing: Does Nirvana Hide Behind the Nebula?) …