Time-Based Reliability of Personality Dimensions®
By: Robert A. McKim, PhD
Overview
As part of the continuous improvement aspect of the PD tool (PD Manual, Appendix C; pp. 2) a time dependence reliability test has been performed that looks at the stability (or reliability) of the testing tool over a period of 18 months. Reliability can be defined as the ability of a testing tool to provide consistent results. Internal test reliability has been measured as a stability index relating the consistency between PD tools (Appendix C equation 11; pp. 13). Time dependent reliability is utilizing the “test-retest” method (http://www.guidetopsychology.com/testing.htm) which provides an empirical basis for measuring the reliability of PD tools over time. This reliability test does not address whether or not individual temperaments change over time.
A control group of 163 participants were given the complete PD assessment suite using all 6 PD tools (Picture Cards; Life Value Cards; At Work Cards; In Relationships Cards; In Communications Cards; and Traits and Characteristics Sheets) in the spring of 2003, and again in the fall of 2004, for a trial period of 18 months. Changes in the PD profile were measured and the degree of change was calculated as a stability index (I).
[1] Ij=(N-δ)/N
where N is the number of the control group participants; δ is the number that displayed any changes between the two assessments; and j is the measure of change. Three measures of change were calculated. These were:
Ip=changes of position i.e. did any temperament type change position
Id=degree of change i.e. how many temperament types changed position
Ic=changes in temperament types i.e. the impact of temperament types on stability position
Test 1: Change of Position
The results for Change in Position (Ip ) are given in the table below:
This is presented graphically (below) clearly showing the top and bottom colour are slightly more stable than the second or third colours.
Test 2: Degree of Change
The aggregate (total) stability index for any changes is 0.23, meaning that there is a 77% chance that there will be some change in the PD profile over 18 months. A better indicator of stability can be see in the graphic below which shows the aggregate stability index for differing combinations of positional changes. In this graphic we can see that there is an 86% chance that the top colour will remain stable and a 69% chance that the top and bottom colours will remain stable
This measure of reliability needs to be applied with care. The PD tool shows a stability index ranging from 0.23 to 0.86 depending on the degree of changes experienced. This test does not indicate which temperament position could change, only that a change has occurred.
Test 3: Changes in Temperament Type
The results for Change in Temperament Type (Ic) are given in the table below.
Data Summary - Change in Type
This information is presented graphically below.
The stability of each temperament type (1, 2, 3, or 4) have not been related to PD preferences (Inquiring Green, Organized Gold, Authentic Blue, and Resourceful Orange), and facilitators should not attempt to relate stability to a specific temperament type. This measure of reliability indicates that there is a probability of 0.83 that reliability of PD rankings of temperament preference is not dependent on temperament preference.
Summary
Over an 18 month period the stability of the PD tool as measured empirically is consistent with the overall accuracy of the tool (< 1 chance in five of an error in the assessment of the top or bottom types). Over the test period there is a probability of 0.86 that the top temperament preference will not change; a probability of 0.83 that the bottom temperament preference will not change; and a probability of 0.77 that changes in the middle temperament preferences will not occur.
The reliability of the PD tool measured in a test-retest situation ranges from 0.83 to 0.86 for the most significant temperament preferences (the top and bottom colours) which is consistent with reliability of other standard temperament measurement tools.
Reliability of the PD tool for independence from error due to individual temperament type is 0.83.
Stability indices for specific temperament types have not been included in this report.