THE RESEARCH PROGRAMME
This has four interacting components:
1. The Measurement Development
Programme
2. Research relating to the Educational System
3. Research on Organisational Systems
4. Research into Societal Management arrangements
1. The Measurement Development Programme
Introduction
Raven's Progressive Matrices and Vocabulary tests were based on observations and theories formulated by Spearman at the turn of the last century.
But Spearman himself noted two crucial defects in these theories.
First, his "general factor" had emerged from studies of the intercorrelations between tests of "educational" abilities - language, science, mathematics, etc. Yet these tests lack both construct and predictive validity - a point developed in more detail by John Raven in his book The Tragic Illusion: Educational Testing.
Second, Spearman noted that the way in which psychologists were trying to tackle the problem of describing and summarising individual differences was basically off-beam. Thus he wrote: "Every normal man, woman, and child is a genius at something. The problem is to identify at what. This must be a most difficult matter in that it occurs only in relation to a small proportion of circumstances. It certainly cannot be done with any of the psychometric procedures in current use."
The absurdity of trying to summarise and report individual differences in terms of two, five, or sixteen variables can be easily exposed by asking "Where would biologists have got to if they had sought to describe all the variation between plants (or animals) in terms of 2, 5, or 16 variables?" Clearly, if we are to improve our ability to describe individual differences and the emergent properties of groups it will be necessary to develop the equivalent of atomic theory in chemistry or a Linnaeus/Darwinian classification in biology.
But the way we think about and try to assess individual differences is not the only problem. Exactly parallel problems arise in the way psychologists tend to think about the environment and the way it interacts with individual characteristics. They again try to "measure" variation between environments in terms of "variables". The problems this poses can again be illustrated by asking: "where biologists would have got to if they had tried to understand and describe the effects of environments on animals by first trying to summarize the variance between animals in terms of 16 variables, then the variance between environments in terms of 10, and then running linear multiple regressions between the two in an attempt to assess the effects of the environments on the animals?" An "ecological" model involving multiple interactions and feedback loops is required.
Both John Raven and his father devoted much of their lives to research in which they sought to advance thinking and assessment in these areas.
In addition to a range of books, this work has resulted in a range of preliminary assessment tools based on an alternative way of thinking about individual differences and behaviour.
Unfortunately, these tools cannot be immediately applied within our current workplaces, educational systems, and public management systems because the operation of these systems is determined, not by personal developmental or societal needs, but by a range of latent, rarely discussed, and hard to influence sociological forces.
But this is not a cry of despair: It points to another topic which has been widely neglected by psychologists: It tells us that human behaviour is not mainly determined by internal properties - such as talents, attitudes, and values - but by external social forces. Such a transformation in psychological thinking and theorising is as great as the transformation Newton introduced into physics by noting that the movement of inanimate objects is not determined by internal, "animistic", properties of the objects but by invisible external forces which act upon them - invisible forces that can nevertheless be mapped, measured, and harnessed to do useful work for humankind.
So this brings us to our fourth conceptualisation and measurement topic: How are these social forces to be conceptualised, mapped, measured, and harnessed in a manner analogous to the way in which Newton made it possible to harness the destructive forces of the wind and the waves to enable sailing boats to get to their destinations?
Dr. Raven would be extremely interested to hear from researchers tentatively interested in working in these areas.
Background
Dr. Raven’s measurement-development activities are grounded in a 40-year programme of research which shows that:
- our education system,
- our organisations,
- our public management system, and
- our economic system
are not only not working well: they are, collectively, heading our species towards extinction at an exponentially increasing rate.
To appreciate the developments that are required in measurement theory and practice, it is really necessary to understand the deep-seated nature of the failures in each of the areas mentioned. However, we will skip over them here but return to them under separate sub-sections of this entry. To go directly to reviews of Raven's research in each of these areas please click on one of the following links:
The Educational
System
Organisational Systems
Societal Learning and Management
Arrangements
More specific observations on the requisite developments in measurement theory and practice are discussed below under the following headings which can be reviewed independently by clicking on the appropriate link.
The Theoretical Bases for the Work
The Conceptualisation and Assessment of Competence
The Conceptualisation and Measurement of Developmental Environments
The Conceptualisation and Measurement of Climates for Organisational and Societal
Innovation and Learning
The Theoretical Bases for the Work
Dr. Raven's research work has highlighted a need for radical change, amounting to a paradigm shift, in the way we think about, not only human resources and institutional arrangements, but also in the way we assess them.
Among the developments required are:
- To refocus our basic way of thinking away from concepts involving "abilities" and "motivation" to thinking in terms of generic high-level, self-motivated, competencies.
- To shift from attempting to describe, or "assess", people in terms of a profile of scores on small numbers of internally-consistent factors, or variables, to making statements (in a form analogous to the descriptions of compounds in Chemistry) about the motives and components of competence that individuals display in specified environments.
- To develop ways of thinking about emergent properties displayed by groups made up of different kinds of people (in a manner analogous to the way one thinks about the emergent properties of compounds in Chemistry).
- To better conceptualise, and find ways of assessing, the nature of the Developmental Environments which engage the motives and interest of a wide variety of individuals and lead them to develop and display a range of high-level competencies.
- To better conceptualise and assess Institutional Arrangements Conducive to Innovation, based on the work of Rosabeth Kanter and Donald Schon.
- To further extend the research in the area just mentioned to thinking about and assessing the Societal Arrangements Required for Innovation and Survival.
The Conceptualisation and Assessment of Competence
A century of work has demonstrated that, so long as we try to work within the mainstream psychometric tradition, it is impossible to get very far. As a title of a review article put it "g and not much else" works.
However, in parallel with this fruitless activity, a growing number (now amounting to some 700) studies of the competencies which distinguish more from less effective performance in a wide range of occupational roles have documented the importance of a range of high-level generic competencies, or motivational dispositions.
The problem is to develop a suitable framework for summarising, thinking about, and assessing such competencies.
Two possible frameworks have been published: One (in Competence in Modern Society) by Dr. John Raven and, the other by Lyle and Signe Spencer (in Competence at Work).
Both are grounded in the framework for thinking about, and assessing, "motives" developed by David McClelland and his co-workers.
Correctly understood, McClelland's measures are not measures of motivation, but measures of the competence to carry out valued activities ... defined in such terms as inventing new scientific theories or putting people at ease.
They measure the respondent's competence to carry out selected tasks by finding out how many of a number of identifiable, cumulative and sustainable, components of competence he or she displays spontaneously while carry out specific kinds of activity.
What this procedure makes clear is that the usual, internal consistency- based, measures of such things as "creativity", "self-confidence", etc. are off-beam: Someone who displays a great deal of creativity when, for example, putting people at ease is unlikely to do so when asked to think of as many uses as possible for a brick.
Pursuit of these insights leads to the realisation that psychologists have probably been misguided in their attempt to classify people in terms of scores on variables (as in physics). They should rather have been seeking to describe people in terms of their motives and the components of competence they display when carrying out valued tasks, in a manner analogous to the descriptive statements Chemists make about substances.
Research in which this alternative way of thinking about competence and its assessment has been translated into practice and has led to some extraordinary reversals of some of the most "well established" findings in educational and occupational psychology.
The Conceptualisation and Measurement of Developmental Environments
In the course of his research Raven and his colleagues have studied the nature of development environments as they occur in homes, schools, universities, and workplaces. To their surprise, a common pattern has emerged.
In developmental environments, parents, mentors, and managers tend to create the following for their children, pupils, students, or subordinates:
- Opportunities to practice and develop important components of competence while undertaking activities which the tutee is intrinsically strongly motivated to carry out. These components of competence include making observations, developing better ways of thinking about things, using feelings to initiate action, monitoring the results of that action, and taking corrective action when necessary, persuading other to help, intervening in wider social and political processes outside the institution concerned, and persisting over a long period of time.
- Opportunities to experience satisfactions which come from the completion of a difficult and demanding activity. It is the experience of these satisfactions which will lead the individual to put up with frustration in order to do similar things in the future.
- Opportunities for the tutee to work with others who, importantly, share his or her basic concerns, values, or motives and (i) share their (normally private) psychological components of competence - e.g. use of feelings to initiate, learn from, and adjust action - while engaged in those activities and in such a way that the tutee can learn from and copy them, and (ii) enable the tutee to see them gaining the very satisfactions the tutee most importantly wants from carrying out those activities.
- Opportunities to gain insights along the lines just mentioned from literature and research-based case studies.
- Opportunities to "try on for fit", or experiment with, alternative ways of behaving in non-threatening situations in which a mistake does not bring dire consequences.
- Placements with mentors who think in terms of multiple talents and try to create working groups made up of people with very different, but complementary, talents in order to create "teams" with dynamic, emergent, properties.
Note how developmental environments engage with the motives or values of individuals: it is as irrelevant to record features of the "environment" of chemical substances which do not engage with the elements of the substance being studied.
What is needed, then, are measures of developmental environments which draw people's attention to these findings, enable them to take stock of the current situation, see what needs to be done to improve it, and monitor progress. The measures need to include questions to find out if there has been a serious attempt to identify individual's motives and talents and redeploy personnel in such a way as to use all available high-level talents for at least part of the time. And to complement these, there is a need for tools to enable managers and others to recognise, develop and utilise the idiosyncratic talents of individuals.
Note that these tools are of crucial importance for the evaluation of of teachers and managers: Have they been able to create developmental environments and climates of dedication and enthusiasm?
The conceptualisation and measurement of Climates for Organisational and Societal Innovation and Learning
As will be shown later, delivery of effective education, health, and welfare services, and, more importantly, the evolution of a sustainable society, is dependent on the evolution of new forms of bureaucracy and democracy. These, in turn, need to incorporate new ways of thinking about the role of the public servant.
We need to charge our public servants with responsibility for creating a ferment of innovation and learning and also hold them accountable for initiating the collection of information, sifting it for good ideas, and acting on it in an innovative way in the long term public interest.
We therefore need tools which can be used to take stock of the extent to which the requisite climates of innovation and learning have been created and, by so doing, direct people's attention to the developments that are needed.
Many of the features which need to be incorporated into such measures are discussed below in connection with innovation in organisations. At a societal level, more attention will have to be paid to:
- the investigation of, and development of strategies to influence, the hidden sociological systems processes which deflect most public-improvement action from the achievement of its goals, and
- the formal arrangements for advancing understanding and especially, for obtaining comprehensive, systems-oriented, evaluations. [Current beliefs about the arrangements to be made to advance understanding are wide of the mark. The only way to move toward a focus on comprehensive, systems-oriented, thinking (instead of on accuracy in relation to isolated specifics) is to provide resources to a wide variety of people who have very different - currently unsubstantiated - perspectives.]
The features to be incorporated into tools designed to direct attention to the key features of new forms of public surveillance of public servants as managers charged with the duties which have been mentioned are less clear. But there is no mistaking the need to move from concepts of government and supervision based in hierarchy to concepts of democracy grounded in fluid, network-based, and issues-oriented arrangements.
Pilot versions of tools to assess both organisational climate (in relevant terms) and public perceptions of society and how it works, including the role of democracy, bureaucracy, and the market place have been developed.