Welcome to My Weblog!

Hakan ErdogmusCheck out the pages on the left bar for information on me. The calendar on the right bar lists my upcoming engagements, meetings and conferences.

To subscribe to my posts, click on the icon at  the top right corner.

hakan_e

Empirical Software Engineering Takes Off

American Scientist magazine has recently published an article by Jorge Aranda and Greg Wilson about empirical studies in software engineering based on the book Making Software: What Really Works, and Why We Believe It (O’Reilly 2010). You can find the article online at: http://www.americanscientist.org/issues/id.13845,y.2011,no.6,content.true,page.1,css.print/issue.aspx.

Prediction Tools for Software Development

At the PROMISE 2011 conference in Banff, Ayse Bener of Ryerson University organized a panel on the future of predictive modelling in software development. Such models are used for quality, productivity, schedule, cost, or profitability estimation and rely on rich historical data, both code- and non-code related, to be successul. Setting them up, collecting the data, and applying the models require much expertise and effort. The economics work only if the results can be acted upon profitably to manage risk, set priorities, and allocate resources. The savings generated must surpass the cost of applying these models. Thus the feasibility of predictive modelling hinges partly on level of automation. That’s where tool support comes into play. As a panel member,  I was asked to comment on this aspect of predictive modelling, focusing on the adoption issue. Specifically, what do we need for widespread tool adoption in this context?

I use the term prediction tool broadly to refer to any implementation of a predictive model or prediction technique, algorithm, or heuristic. I include in the definition any facilities that perform essential input functions such as data identification, extraction, and sanitization, as well as essential output functions such as reporting, summarization and presentation.

Types of Tools

Use of prediction tools is very rare, although we know of several successful examples. I’ll contrast two typical manifestations of such tools:

  • A: stand-alone, one-of-a-kind, intermittently applied facilities (usually collection of scripts and small applications) often intended for use by experts who have developed or commissioned them, and
  • B: stable, deployed components, applications, or services that are part of an existing software development environment and meant for regular use in software teams.

Manifestations that are a mixture or fall somewhere in between exist, but they won’t be useful for making my point.

Tools of type A are most common. Almost all of the noteworthy examples I know of fall under this category. They are relatively cheap to develop, but each application requires expert hands. They are meant to be used by a small minority of very select people.  That alone pretty much explains their home base of large organizations with dedicated metrics, research, or process improvement departments. Indeed, at a session of the International Software Engineering Research Network‘s annual meeting this year (which happened to collocate with PROMISE), we have seen some pretty nifty examples from ABB and Avaya Labs. Type-A tools are thus fine for a large organization where the cost of the required expertise can be amortized over big, corporation-wide initiatives, however they are too resource-intensive for broader adoption.

The alternative to this “do your magic and throw the results over the bench” approach is type-B tools, in which higher up-front development costs by those handful of experts can be amortized across many organizations over more frequent use by multiple roles in software teams themselves. The scope of each use may however be small. This attractive alternative doesn’t necessarily imply that no fine-tuning, hand-holding, or maintenance will ever be needed, but it does imply making information more available and friendly to software teams and decision makers, thus giving them more control over their projects and achieving better visibility.

So let’s limit the scope to type-B tools, say because I’d like predictive modelling to be more readily available to mere mortals. What factors should we look at next to increase the chances of adoption? I’ve organized these factors in two dimensions:

  • factors specific to the goals of prediction, and
  • factors that encourage tool adoption in general.

Continue reading Prediction Tools for Software Development

Don’t miss SES ’2011

Following up on the smashing success of its first edition in Munich, Germany, IEEE Software is organizing the second edition of Software Experts Summit, this time at the Computer History Museum in Mountain View, California. Both the venue and the line of speakers are pretty amazing. Registration cost is $90. Featured speakers include Gary McGraw, Jan Bosch, Grady Booch, Grigori Melnik, Linda Rising, Pekka Abrahamsson, and Rebecca Wirfs-Brock.

For program and registration visit SES ’2011 page at http://www.computer.org/ses11.

Protected: Siemens India Training Materials

This post is password protected. To view it please enter your password below:


How to Evaluate Cost-Effectiveness of Software Projects and Process Improvement

Today I finished a training course for Siemens India on cost effectiveness of software development projects and process improvement initiatives. The course was conducted over LiveMeeting in 5 consecutive days. 23 people from 4 different locations participated in the training. We covered basic techniques for evaluating cost-effectiveness, such as NPV and Real Productivity, and worked out several examples. Part I focused on Project Valuation and Part II focused on Process Improvement. Each day’s session started with a check-in  that addressed outstanding items, reviewed concepts, and discussed solutions to take-home exercises from the previous session.

As the common context, I used that of a mid-sized organization in the health informatics sector with new ventures in its project pipeline and with several new ideas for improving its software processes. The examples were mostly drawn from this context and organized as HBR style cases. The cases addressed project staging, incremental development, pair programming, code inspections, role of evidence, and test-driven development.

The event was a repeat of an earlier course I gave in June for for Siemens in Erlangen, Germany. But I changed the materials and format significantly to fit the new  interaction style and  in response to the feedback I have received after the Erlangen version. I divided the material into 5 modules, incorporated smaller examples, and added take-home exercises for each module. I also frequently used LiveMeeting’s poll feature to engage the audience. All in all, I think the format worked very well, surprisingly with very few technology glitches. The tech support supplied by host was very good.

The course description is available here. If you were a participant and haven’t filled out the course evaluation form, you can find it here. I have opened this post for comments: if you have any, well, what are you waiting for? Do fire.

The Value of Process

In A Process That is Not, the November/December edition of my IEEE Software column, I tell the story ofTikiWiki.    This is an interesting and telling story for those of us who preach software process.

A related point of view is provided by James Bach in an early article he published in IEEE Software in 1995. See Enough About Process: What We Need Are Heroes. In that article, Bach defines process heroism as:

I’m fascinated by the fact that people who take initiative to solve ambiguous problems are key to the success of every software project. I call this software process heroism, based on Joseph’s Campbell’s description of the hero’s journey. Unfortunately, it’s a subject that provokes a lot of strong feelings and confusion. There seems to be a lot of fear surrounding the simple idea that people are not easily controlled, influenced, or interchanged with one another.

I think this controversy is the single biggest indicator of how immature our field is. We can’t even agree on what we are doing or what we mean, as people, in the midst of our process creations. I believe we will never be able to control software projects well, unless we give up on treating software development as a mechanical task, and start embracing it as a human system. Anyway, that’s what works for me.

Hmmm. Food for thought…

hakan_e

Architecture Meets Agility

In June 2009, the Software Architecture Challenges in the 21st Century Workshop sponsored by IEEE Software, University of Southern California, and University of California Irvine attracted quite a bit of interest. Pekka Abrahamsson, Philippe Kruchten, and Grady Booch all addressed the role of software architecture and architecting in agile software development. I ended up writing an editorial based on the Kruchten and Booch talks for the September/October 2009 issue of IEEE Software.  The editorial is also featured in the August 2009 edition of Computing Now.

In that editorial I explore the common themes raised by Booch and Kruchten, and also touch upon where the two gurus differ. The common themes I identified are:

  • The scope of software architecture
  • Architecture-phobia
  • Architecture’s omnipresence
  • Architecture for self-governance
  • Architecting as a continuous activity
  • Just-enough architectural representation

These themes made me reflect on my former views on the topic of how agile and architecture interact. Those views were expressed in a tongue-in-check article that I had written for Agile Times back in 2003 .  The article was sarcastically titled “Let’s Scale Agile Up”. In those relatively early days of agile, I used to advocate a purist agilista approach that downplays architecture. I used to object to any attempts to scale up agile software development by augmenting it with explicit architecting. I was wrong. I realized how wrong I was in a recent project.

The project was small, but I couldn’t crack it with seat-of-the-pants, “let the design emerge” attitude. The project’s success relied on the ability of a complex core technology to solve the problem at hand. We had to nail down technical decisions at the beginning. And these decisions couldn’t be reversed easily later, even though the project was small.  My client demanded to know about the main components, how they would interface with each other and be integrated with the core technology, how they would be decoupled from each other for maintainability and extensibility,  and what general functionality they would provide. He wanted me to document the underlying decisions. In a nutshell, he demanded that I do some architecting.

I ended up spending a week and a half on these issues, constantly soliciting clarifications from my clients on the requirements, running endless spikes to resolve technical risks pertaining to the core technology, and coming up with a general design in the process. And I am glad my client insisted because this upfront activity paid off.  It turned out that seemingly subtle aspects of the requirements had an impact on how the core technology, a hairy rule engine, would work to solve my client’s problem. Interestingly, I no longer associated all this early activity, as I did back in 2003, with the “grit” between the gears, the grit that contaminates the project and brings it to a screeching halt.  That grit did not interfere with planning and running the project as a highly incremental and iterative one. Actually, it turned out to be a lubricant rather than a contaminant. I grew up, I suppose.

hakan_e

Who are Superprofessionals?

No, they are not superstars, or all-knowing super-powerful CEOs poised to control the world. Although one reader, and letter writer, thought that’s what I meant in my latest IEEE Software editorial titled The seven traits of superprofessionals.

The letter writer states that he has been researching the reasons of  the recent economic crisis and states that he’s found a strong indicator of what kind of management behavior the future requires. Apparently, it’s not exactly what my editorial describes (never mind that my editorial was not about management behavior at all). The letter writer attaches to the term “superprofessionalism” — which I coined to describe a set of behavioral traits that represent a sort of high-order professional conduct — the same connotation as the false, shallow stardom associated with the kind of people featured in reality TV series like Supermodel, The Apprentice, or  The Bachelor.  (As a synopsis, the traits mentioned in the editorial are: focus on individual responsibility, acute awareness, brutal honesty, heightened sense of fairness, resilience under pressure, attention to detail in perspective, and pragmatism first). He alleges that those kinds of media programs and my article represent the last dying attempts to revive an old-economy thinking in which all-knowing individuals hold and dispense all power. He adds that my article advocates individual and central decision making at the expense of group decision making, and is counter to holistic team behavior and denies the emergence of information and collective intelligence from decentralized interactions in social networks. But the best stretch is when he puts the blame for the current economic crisis on the kind of thinking my writing encourages, like, say brutal honesty (he picks on this trait in particular, stating that it’s socially unacceptable).

Oh? Interesting perspective, although it completely beats me how any of that can be inferred from my editorial. If it could, that perspective would represent a truly curious internal paradox given my links to the the agile software development community, and in light of my previous column on The role of diversity in software development.  But one can never underestimate the intellectual depth of those who can with great facility read the sub-sub-subtext between the lines. Honestly, can anyone detect any nuances in that writing,  implanted with the idea to recreate or preserve old-world power? And me at the center of the conspiracy?

Cheers

hakan_e

Top Ten Ways to Kill a Paper – Part II

In a previous blog, I talked about five common mistakes authors make when trying to get an article accepted for publication in peer-reviewed software engineering journals, magazines, or proceedings. The full list includes ten common mistakes, as follows:

Number 10. Ignore reviewer feedback.

Number 9. Stick to clichés.

Number 8. Borrow without giving due credit.

Number 7. Don’t worry about the language.

Number 6. Ignore your work’s limitations.

Number 5. Ignore the context and prior art.

Number 4. Rely on argumentation to push an idea.

Number 3. Ignore your audience.

Number 2. Be undeservedly authoritative.

Number 1. Write in a purely descriptive style.

Continue reading Top Ten Ways to Kill a Paper – Part II

Cloud Computing

The February issue of Computing Now has a nice collection of articles on cloud computing. See http://www2.computer.org/portal/web/computingnow/0209/theme/software

My IEEE Software column on the topic is also featured (Cloud Computing: Does Nirvana Hide Behind the Nebula?) …

hakan_e