Equality is Overrated
It has always bothered me that a lot of project quality or risk "assessment" tools or questionnaires automatically assume that each question is equally weighted. We all know that some of the questions are more important than others but most tools are designed to treat the questions equally. Even when they are not equal, often the relative importance between the questions has no credible basis. I have found that users of these tools often do not question the assumption that the questions are equal.
When this "equal assumption" is invalid it can lead to justifying decisions that may not be in the best interest of the project or organization as a whole.
It was with great excitement that I facilitated a group of Information Technology leaders of Project Management Offices from some of Central Florida's most prestigious companies to evaluate questions of a risk tool. The goal of the evaluation was to establish the relative importance of each question in a risk questionnaire. I will be formally sharing all of the results of the tool development in a future publication but here are four items of interest that drive home the point that all questions are not equal.
1. Even though there is a lot of focus on the challenges of managing globally and co-located teams in comparison to other risk factors it was not important to the PMO leaders. (Note most of these leaders are from companies with a global reach) Specifically...
Scope Maturity was 23 times more important than Co-located teams as a risk factor.
2. Even though there is a lot of emphasis on methodology, the lack of a methodology was not seen as a risk factor in comparison to other project environment variables. This from PMO leads!!! In fact...
having an experienced, capable project manager was 7 times more important than a methodology.
3. The number one risk factor was "Scope Maturity" with the question being weighted at 26 percent of the total score. The number two risk factor was "Change Readiness" (The customer and stakeholders willingness to accept the change) being weighted at 20 percent of the total score. These two factors dominated other factors perceived as critical such as "Technical Complexity" which only warranted 2 percent of the total score. These PMO leaders were essentially saying...
if we understand the scope and the organization is ready for the change and the project manager is capable we have it covered!
4. I have used this tool in my last two project risk management classes for participants to evaluate their project with both a weighted and un-weighted questionnaire and a lot of them have found their projects scored at a higher risk level with the weighted questionnaire. Stated another way... the un-weighted risk questionnaire could be masking risks.
There was a lot of lively and valuable discussion as the relative weights were determined with the PMO leaders. The results presented here are an average of the group and group members did have differences in their weights. The next step in the maturation of my risk questionnaire tool will be refinement and assessment by a group of CIO's. In the meantime be cautious about equality because few things are truly equal.
Dr. James T. Brown, author The Handbook of Program Management - McGraw-Hill
Copyright 2010 SEBA Solutions Inc. All rights reserved