Developing a New Social Science Tool Using Engineering Project Delivery Methods
By: Rob McKim
Normally the worlds of the social sciences and engineering do not intersect smoothly. Engineers tend to ignore that fact that humans build, and will ultimately use, the artifacts produced through engineering projects; and social scientists tend to underestimate the problems associated with inanimate objects and overestimate the degree of information available from physical tests.
In 2002 I was approached by Denise Hughes of Career life Skills Inc. to help develop a temperament based assessment tool that would bridge the social/hard science gap. I am a practicing civil engineer and researcher and I am very familiar with the scientific method, and project management delivery systems for physical systems. I am also familiar with several existing temperament-based tools, and through long exposure to these tools I was convinced of their efficacy in engineering situations, even if the supporting data was quite vague by engineering standards. I have had challenging experiences when trying to utilize social science based tools in an engineering environment, and to tell the truth I was not sure this bridge was possible
There is a great need in the engineering world for people-based tools, and engineers are perhaps the hardest people to convince without “hard” data. Denise wanted to develop a temperament-based tool that could be defendable to hard science types, just like me. I undertook the project on behalf of CLS with some degree of hesitation, and only after I was assured that I would have a free hand in the development of the experimental and analytical aspects of the process, and that we would utilize well established project management techniques. This was agreed to and the project was underway in the spring of 2002.
The first step in any project, social science based or otherwise, is to develop the Project Life Cycle. All successful projects follow the well-defined series of phases shown below.
Establishing of the project-life-cycle is an essential planning tool for any project; it ensures that elements are executed sequentially and that adequate control exists.
The concept stage is the most critical in the PLC. This stage sets the scope and goals of the project. The cost of project control is optimized at this stage where scope changes can easily be made with little loss of time or resources. This is demonstrated in the following figure. We see that as we move through the PLC the ability to control or change the project decreases exponentially, and the cost or impact on resources to make changes rises exponentially. Clearly we need to maximum the impact at the concept stage.
Specific tasks and deliverables were clearly identified for each phase of this project.
In this phase I firmly established the theory (hypotheses) that would drive the rest of the project. The main hypotheses were that the are (A) 4 major categories of temperament, (B) that these temperaments can be measured, and (C) that temperaments can be considered as dependent variables for some behaviours. These hypotheses were supported by a sound literature search that looked at existing temperament tools, classic psychological theories, and expert opinion.
The development phase of this project was to utilize the confirmed hypotheses (from the concept phase) and develop a tool based on the scope set in the concept phase.
The development phase took the hypotheses that were verified in the concept phase and developed appropriate tools to utilize the hypotheses. 6 sorting tools were developed to correlate the dependent variables of temperament to individual choices. The 6 tools were depictive picture cards, verbiage describing at life situations, at work situations, in communication situations, in relationship situations, and specific traits and characteristics. The 6 sorting tools were tested in focus groups consisting of 112 people from 5 provinces. A statistical analysis was completed on the 6 sorting tools against the hypotheses. Sorting tools that demonstrated statistically low correlations between the independent variables and the dependent variables (behaviours) were reworked.
The final phase of the project was to execute, or implement the tool in Canada. The implementation followed a 4-step process, and is currently underway. The first step was to make participating facilitators familiar with the tool, and to obtain certification for the new tool. This was partially accomplished during the focus group sessions in March and April of 2003. The focus groups had a triple purpose; the first to provide feedback from the development phase that produced the 6 temperament sorting tools; the second purpose was to allow known temperament based facilitators to try the tools, and ask questions of the direct authors; and the third was to allow a certification process whereby facilitators could obtain national level certification in the new tool.
The second execution step was to utilize the feedback from the focus groups to upgrade the tool to reflect both technical and personal observations about the tool. Valuable information was collected during the focus groups concerning the format, style, and execution of the tool. Based on this feedback many aspects of the tool were modified.
The third execution step was to provide information and certification sessions for non-focus group participants. This was accomplished by a series of (ongoing) seminars across Canada where the completed tool would be introduced to facilitators and certified as to its use.
The last step in the execution is underway, and includes the full-scale implementation of the tool. This included the distribution and production of the tool for facilitators. Included in the distribution package are: (1) Facilitator certification criteria, (2) Resource manual including theoretical backup (for the engineers), (3) Facilitator’s guide (electronic and hard copy formats), and (4) the tool itself.
The execution phase also includes a continuous improvement component for the tool that monitors and upgrades the tool as it is used. This continuous improvement element requires that facilitators provide both statistical and opinion-based feedback from their sessions that will ensure that the tool is always up-to-date.
The worlds of engineering and social science are not as far apart as many believe. Application of proven engineering based delivery systems to a social science based tool development can work if both the social scientist and the engineer take the time to look over their respective fences and realize that there is no need to reinvent the wheel. There is really very little difference between building a bridge and building a social science based tool.