%i,*jM l»tate (HfoUcge of Agticulturc Kt (StOtnM 5llni»eratt8 3tliata, 5J. S. Cornell University Library LB1131.B65 Improving schools by standardized tests, 3 1924 013 378 561 The original of tliis book is in tlie Cornell University Library. There are no known copyright restrictions in the United States on the use of the text. http://www.archive.org/details/cu31924013378561 IMPROVING SCHOOLS BY STANDARDIZED TESTS BY SAMUEL S. BROOKS District Superintendent of Schools, Winchester, N.E. UNDER THE EDITORSHIP OF B. R. BUCKINGHAM, PH.D. Director of the Bureau of Educational Research Ohio State University HOUGHTON MIFFLIN COMPANY BOSTON NEW YORK CHICAGO SAN FRANCISCO COPYRIGHT, 1922 BY SAMUEL S. BROOKS ALL RIGHTS RESERVED CAMBRIDGE ■ MASSACHUSETTS PRINTED IN THE U.S.A. EDITOR'S INTRODUCTION This book divides itself naturally into two parts. The first part has to do with the situation in which Superin- tendent Brooks found himself, with his successful campaign in educating his teachers to use standardized tests, with the results which he obtained, with the way he used these results to grade his pupils, to rate his teachers, and to evaluate methods of teaching, and finally with the use he made of intelligence tests. This is the first time, so far as I know, that a practical school man has, after planning carefully and executing to the last essential detail a large testing program, set down for the guidance of other prac- tical school men just what he did. We have waited a long time for this. The test-makers and technical research workers have been heard from exten- sively; and some of them have written for the benefit of teachers and supervising officials. Here, however, is one of their number who found unfavorable conditions in the district to which he had been assigned. He secured the support of his teachers in making a survey of these con- ditions by means of standardized tests. He gave the tests three times and made certain important uses of the results. Teachers, pupils, and parents were in favor of the tests as he used them. Moreover, he tells us just how he did these things; and he tells the story so clearly and so vividly that any one who reads his account will feel that he can do likewise. The second part of the book has to do with the changes iv EDITOR'S INTRODUCTION in methods of teaching which were brought about because of the knowledge gained from the tests. This is the best — is it not the only? — account of a testing program which has been carried through to its logical conclusion. It is the best answer I know to the question so often raised, "What shall we do with the results of tests?" Nor is the answer conceived narrowly. It is not concerned with mere devices. It has a philosophy, and it is broad and fundamental. Definite methods are shown and shown in detail, but they are based on sound general principles. For example, he justifies his startling doctrine that the first reading in- struction should be in silent reading by the principles of association. Again, he introduces a plan of teaching chil- dren how to study because the tests measure the results of study and because if one wishes to improve the results of teaching as measured by tests, one must first improve methods of study. Even with reference to particular subjects the measures he has adopted and described are something more than expedients. Even here they are broadly conceived. In reading, for example, he recognizes a general principle ap- plicable to all subjects. The general principle is that in order to improve instruction in terms of test results, one must develop in the pupils the abilities which the tests measure — not other abilities, not even similar abilities, but the precise abilities which the tests measure as nearly as these can be ascertained. I am aware that there will be a few transcendentalists among research workers who at this point will cry out that neither Superintendent Brooks nor I know what the tests measure. These are the men whose daily occupation is EDITOR'S INTRODUCTION v straining at gnats and swallowing camels. It is well enough for them to do this among themselves, but some of them would hold up all progress while they split hairs over the setting up of a theoretically perfect "criterion" and the devising of a test of highest possible correlation with it. This is good work, and I do not deprecate it. But the work of the schools must be done. Instruments of measure- ment must be used even if they are lacking somewhat in validity. At any rate, a practical man who thinks about silent reading readily concludes that rate and comprehension are the two items which ought to be measured and which the tests do apparently measure. He infers, therefore, that a method of teaching reading which is to develop the abil- ities measured by the tests must aim at these two objec- tives — rate and comprehension. This at least is Super- intendent Brooks's line of thought, and he pursues it with vigor and success. From the point of view of the expert in tests the author xmdoubtedly displays an uncritical acceptance of them. Fine statistical points are ignored; practice effects are for- gotten; validity is assumed; and reliability is all but un- questioned. In short, the test-maker is accepted as hav- ing done his work. Some of us are sure that he is not en- titled to this degree of confidence; nor — to be entirely fair — does he think so himself. For the most part he is an experimentalist, trying one device after another, call- ing this a reading test and that an arithmetic test, with- out agreeing with his fellow test-makers as to what ability in either of these subjects may be. There is in this book an impressible object lesson for the test expert. From it he may vi EDITOR'S INTRODUCTION see clearly how vital it is that he analyze the objectives in teaching and that he make his tests true measures of the extent to which the objectives are realized. It is clear that if school people are going to adopt the tests in the spirit displayed by Superintendent Brooks's teachers in chap- ter X, a burden of responsibility is laid upon the test- maker which he cannot evade. Again, there will be many practical teachers, principals, and supervising officials who will feel that there are many products of teaching which the tests fail to measure. These school men and women will point out that the abilities which are here exalted are of the more formal type. Yet it is entirely possible that the abilities which the tests directly measure are representatives of the higher abilities which the tests do not directly measure and which it is un- doubtedly the business of the school to inculcate. For example, rate and comprehension of reading, which the tests purport to measure directly, may fairly be said to condition the acquiring of more complex qualities having to do with taste, attitude, and appreciation — qualities which manifest themselves to an increasing degree as rate and comprehension of reading are developed. It may very well be that the better our pupils can read, add, and spell, or an- swer questions in language, geography, and history, the better they will manifest the qualities which are valuable in a social organism. Indeed, it can hardly be otherwise. These subjects are in the course of study by common con- sent; nor are the efforts of the curriculum makers, even of the most advanced type, directed toward their entire withdrawal. They are in the course of study because they are believed to serve the higher purposes of education. If EDITOR'S INTRODUCTION vii this is true, there is a case to be made out for teaching these subjects well, and for the belief that, within reasonable limits, the better they are taught, the more surely the broader purposes of education will be served. It will be time for the experts in high places and the conservatives in all places to criticize the author when they have accom- plished as much without tests as he has with them. To my mind chapters x to xv — constituting what I have called the second part of the book — are unique. Professor Cubberley has pointed out that because of the measurement movement, the whole subject of Educational Administration, in a decade or two, may need to be reor- ganized and books on the subject rewritten in terms of this new scientific development. The same statement may be made as to the subject of Methods of Teaching and as to books on that subject. Those methods will gain favor which produce measurable results. A method will not be taken for granted because some one high in author- ity favors it. In fact in these days no one high in authority will promulgate a method merely as his ipse dixit. Super- intendent Brooks has much to offer in these last chapters about method — methods of teaching reading in both pri- mary and upper grades, methods of handling the content subjects and of teaching children to study. But these methods are the immediate outgrowth of his testing pro- gram, and they are designed to improve the conditions which his testing program disclosed. They are methods which he has tried and the effects of which he gives us. They are therefore convincing. B. R. Buckingham PREFACE This book is written for superintendents, teachers, and all other persons interested in the use of educational tests and measurements as a means of improving the work of the schools. It tells how standardized tests and scales were used periodically for two years throughout an entire su- pervisory district, and how the results of the tests were put to practical use in classifying pupils into grades, for meas- uring the progress of pupils in their studies as a partial basis for promotion of pupils, for measuring the effi- ciency of teachers, for measuring the relative efficiency of special methods, and for motivating the work of both teachers and pupils. It also teUs of remedial measures, in the way of modification of subject-matter and teaching methods, taken to improve unsatisfactory conditions re- vealed by the use of the tests. In its field, the book is unique in at least two respects: (i) it is a narration of actual experience rather than an ex- position of theory, and (2) it describes in detail how a com- prehensive, periodical testing program was planned and carried out, how the interested cooperation of the teachers was secured, and how the results of the tests were used to improve the schools. One whole chapter is devoted to what the teachers and pupils had to say, at the end of two years' use of the tests, as to how the tests were of help to them in their work. So far as the author is aware, all the books previously X PREFACE published on the subject of standardized tests and measure- ments deal almost entirely with the tests themselves; the need for them, descriptions and reproductions of them, and data concerning their derivation and standardization. Very little is said as to how the tests can be put to practical use in the schoolroom, and that little is usually expressed in very general terms. No book heretofore pub- lished describes in detail the putting of standardized tests to practical use in a concrete situation and periodically over a period of years. The above-mentioned books have proved very valuable in making the educational public acquainted with stand- ardized tests and their possibilities. But is it not time that we had something definite in book form as to how the tests are proving of value, or otherwise, in the hands of teachers and administrators? It was with the idea of starting something in this line that the author has set down in this book his own experiences with standardized tests as an aid in school supervision. The book does not claim to settle any of the great problems of modem edu- cation. It merely tells how standardized tests were used in an attempt to solve some of the more pressing of these problems as they appeared. In it we tell what we did, why and how we did it, and the results obtained. We have tried to tell it in such a way that any one who so desires may easily follow the general plan with modifications, if necessary, to fit his own particular field of endeavor. It is hoped that the book may prove both interesting and help- ful to our fellow workers, and that it may give at least a little added momentum to the great movement toward ob- jective measurement of "classroom products." If it ac- PREFACE xi complishes these objects, even in a small measure, the au- thor will feel amply repaid for the time and labor involved in presenting his experiences for publication. It is with pleasure and gratitude that the author here ac- knowledges his indebtedness to Dr. B. R. Buckingham for kindly advice and encouragement during the preparation of the work, for many helpful suggestions, for his sympa- thetic editing of the manuscript, and finally, for the Editor's Introduction. S. S. Brooks Winchester, N.H., July, 1922 CONTENTS Chapter I. Introduction . . . . . 7 7,". i Chapter II. The Practical Situation 8 Sizing-up the situation — Analyzing the situation. Chapter III. Getting Teachers to Feel the Need for Standardized Tests i6 The first meeting — The second meeting — The third meeting — The foturth meeting. Chapter IV. Using Standardized Tests for Grading Pur- poses 30 Supervising the testing — Converting scores into grades — Devising a graph card — The meaning of the graph card — Grading the pupils. Chapter V. Conditions Revealed by the Use of Standard- ized Tests 39 The graph cards described — Tendencies shown by the graphs — The search for causes; over-emphasis on arithmetic — Other tendencies — The writing situation — Poor results in reading, and why — A new policy as to reading. Chapter VI. Measuring the Progress of Pupils by Means of Standardized Tests 52 Teachers' judgments of progress unsatisfactory — Standardized tests used to measure progress — The graph card — The record of a child of average mentality — The record of a bright child — The record of a dull child — The record of a class. Chapter VII. Measuring the Efficiency of Teachers BY Standardized Tests 69 Factors in teaching ability — Measuring teaching efiBciency by results — Objections of teachers to rating by results — The objections answered — The plan of rating teaching — First illus- tration of the plan — Second illustration of the plan — Salary and rating. xiv CONTENTS Chapter VIII. Comparing the Efficiency of Spectal Teaching Methods by Means of Standardized Tests . 82 The teacher and the method — Eliminating the variables — Two ways of comparing methods; the first case illustrated — An ex- periment in comparing methods — The second case: comparing methods when -used by the same teacher — A suggested plan of procedure '— Need of testing methods by results. Chapter IX. Some Uses FOR Intelligence Tests ... 96 Practical notes for practical purposes — Intelligence tests needed in school: faulty judgments of teachers — The fallacy that all pupils can make satisfactory progress — The Binet-Simon Intel- ligence Scale — Group tests of intelligence — Our original pur- pose to locate the mentally defective — The Otis Test — The Haggerty Tests — A case in which intelligence testing helped — Another case — A menace — Border-line cases — Intelligence tests used in rating teachers — Identifying the bright pupils — Special opportunities in city systems — Intelligence tests for grading purposes — The small range of mental ages in each grade — Mental ages of pupils in each grade — Grading by mental and achievement tests substantially the same — A pro- posed plan of grading in a rural school. Chapter X. Reaction of Teachers and PTn>iLS to Stand- ardized Tests 124 What the teachers think of the tests — What the pupils think of the tests. Chapter XI. Reading Aims and Methods 143 Modification of methods and materials is part of a testing pro- gram — Drill in oral reading does not ensure silent-reading abil- ity — The way to improve silent reading is to teach silent read- ing — - Oral-reading drill hinders good reading — The case against oral reading — Yet oral reading has its value — Oral reading not necessary for beginners — When to begin to teach oral reading — Why silent reading should be taught first — Silent reading as actually taught — Merely giving children books to read is not sufficient. Chapter XII. Silent Reading in the Lower Grades . .159 Must develop the kind of ability the tests measure — The success of the plan adopted — Reading in first grade — Teaching the first words — Keeping the right order of association — Teaching CONTENTS XV sentences — The use of pictures — Reading from boolcs — Little good material available — An example of usable material — One way of using such material — A second method — A third method — Second readers begin in the last half of the first year — The result in one school — The kind of material needed for first-grade use — Reading in the second grade — Reading in the third grade. Chapter XIII. Reading in the Upper Grades . . .193 Reading as a separate subject should not be necessary in the upper grades — Local conditions required vigorous action — The plan — Character of the reading material — The boolis mentioned are merely suggestive — Why literature is not adapted for silent- reading drill — The kind of material needed for silent reading. Examples of good selections for silent reading — Ways of using silent-reading material. Chapter XIV. Teaching Children How to Study . . 234 Children do not know how to study — Children must be taught to study — Poor methods prevail — Situations "which favor good study habits must be provided — Good silent-reading ability essential to study — Good silent-reading methods encourage good study habits — Some examples — Habits thus developed carry over into other work. Chapter XV. Supervised Study 246 Three types of activity in "studying" — Difficulties in connec- tion with supervised study — Silent-reading drill offers an oppor- tunity for supervised study — The question method is especially advantageous — The method illustrated — Alternating super- vised study and recitation — Another way of conducting super- vised study — Finding the topic of a paragraph — Construction of outlines — Collecting material as a phase of study. Index 275 LIST OF TABLES Table I Distribution of the Ratings of a Sixth-Grade Arithmetic Paper 17 Table II. Distribution of the Ratings of an Eighth- Grade History Paper 18 Table III. Weights assigned to Twelve History Ques- tions according to Teachers' Judgments .... 22 Table IV. Rank of History Questions in Difficulty based ON Number of Times missed 24 Table V. Summary of Ranks and Values for Each of Twelve History Questions 25 Table VI. Record of L. D 35 Table VII. June Scores of a Fifth Grade in Reading . 75 Table VIII. Grade per cents on Each Test — Teacher A 76 Table IX. Grade per cents on Each Test — Teacher B . 77 Table X. Grade per cents on Each Test — Teacher C . 78 Table XI. Grade per cents on Each Test — Teacher D . 79 Table XII. Average Scores in the Woody Scales . . 89 Table XIII. Comparison of Mental and Chronological Ages of Sixth-Grade Pupils, Tamworth Schools . .115 Table XIV. Mental Ages of Pupils in Each Grade . .117 Table XV. Results of Mental Tests in a Selected School i 20 Table XVI. Pupils with Mental Ages below Eight Years 120 Table XVII. Pupils with Mental Ages between Eight AND Nine 121 Table XVIII. Pupils with Mental Ages between Nine AND Ten 121 Table XIX. Ptn-iLS with Mental Ages between Ten and Eleven 122 IMPROVING SCHOOLS BY STANDARDIZED TESTS • CHAPTER I INTRODUCTION Education is gradually becoming less of a custom and more of a science. A force of professionally trained edu- cators, imbued with a determination to free the schools from the chains of mediaeval traditions in subject-matter and method, and with an ambition to apply the principles of common sense to education, is being rapidly built up. The development of modem educational psychology is bringing about changes of management and method that are little short of revolutionary. The ossifying doctrine of formal discipline and a multitude of other hoary tradi- tions have been proved invalid by the scientifically con- trolled experiments of experts in the psychology of learn- ing. Thus the very foundations have been knocked from under the existing educational system, leaving it, so to speak, very much up in the air. But if these experts were to be anything more than iconoclasts, they must furnish something better in the way of teaching aims and methods to replace the demol- ished idols of an educational world whose complacency they had sadly ruffled. Accordingly, they inaugurated in some quarters experimental schools and school systems — laboratories for the compounding of educational doctrines. While still very young, however, the new movement be- came seriously afflicted with a rash of faddism due to 2 STANDARDIZED TESTS under-trained but over-enthusiastic reformers who mis- took their day-dreams for facts of proved worth. Having passed through this period, however, the movement began to steady itself. Earnest seekers after the truth early came to realize that to foist half-baked schemes on an ultra-conservative public would in the end merely breed distrust and suspicion of all new ideas in education regardless of their value. Reformers grew more cautious of presenting their ideas until they had proved their worth. Local school men and women also became more cautious of introducing new methods until, those methods had been successfully tried out under fairly representa- tive conditions. Thus something like a healthy balance was struck between faddism and conservatism. Among the more recent ideas there are some that seem pregnant with tremendous possibilities for the improve- ment of educational procedure. The most far-reaching in its beneficial possibilities seems to me to be the idea of measuring the results of teaching and the intelligence of school children by means of standardized tests. Of large, importance also are: " The Socialized Recitation," " Supervised Study," " Teaching Children How to Study," "The Project-Problem Method in Teachmg," , and the change of emphasis from oral to silent reading. All these things are rich with promise of greater efl&ciency in education. But although all the above-mentioned ideas have proved successful, when carried out by trained men and women full of enthusiasm and thoroughly under- standing the psychological principles upon which the ideas were based, nevertheless, when the carrying-out of these same ideas has been attempted by persons lacking in sympathy and fuU understanding of the principles, aims, and methods involved, they have often been far from successful. INTRODUCTION 3 Superintendents and teachers who condemn these ideas, after proving to their own satisfaction that they are worth- less, remind me of the first farmer in the town of my boy- hood to purchase a mowing machine. For a great many years this man had mowed his many rough and rather rocky acres by hand with the help of a large crew of hired men. But one spring the mowing-machine agent appeared in our town and as a result, by the time hay was ready to be cut, the farmer found himself possessed of a brand-new mowing machine, resplendent in green, yellow, and red. The next morning he started out to mow the " back field." Truly his trials that day were many. And when the machine, imable to stand any more of his inexpert han- dling, broke down late in the afternoon, he had succeeded in getting over only about as much ground as one good man with scythe and snath could have mowed in the same length of time. And what an untidy-looking job it was! The next day the farmer started his old crew to mowing by hand. No more of that mowing machine for him. He could do more and better work in a day alone than could a man with a pair of horses and a mowing machine. The mowing machine was a failure. He knew because he had tried it. But did this farmer's experience with his new mowing machine prove the failure of the machine or the failure of the man? The mowing machine, we know, will do good and rapid work in astonishingly rough and rocky ground when guided by a practiced hand. And so it is with new methods and ideas in education. The socialized recitation can easily result in chaos in the hands of weak teachers. Supervised study can become the sepulcher of all energy and initiative on the part of pupils if improperly conducted. Teaching children how to study becomes a farce when attempted by teachers who do not themselves know how to study efficiently. And 4 STANDARDIZED TESTS how many of them do know how to study to the best advantage? As for silent-reading — well, some of its ardent exponents would get the surprise of their lives if they sent out questionnaires to all the teachers of so-called silent-reading asking them to explain their conception of silent-reading drill. For example, I visited a village school not long ago, where, the superintendent told me, one of his best teachers was working out some of the " new ideas." I found the teacher industriously correcting papers at her desk. The room was very quiet and orderly, and every child was busy. "Do you teach silent-reading here?" I asked, after introducing myself. " Oh, yes! That is what we are doing now. We have silent-reading drill for a whole hour every afternoon." " And is that all the silent-reading drill they get?" She looked up in surprise. " Why, yes, that seems to be as much time as we can spare without neglecting the other work. The pupils enjoy it very much," she added. And why should n't they enjoy it? The school was well suppHed with children's story-books and for a whole hour every afternoon the pupils were permitted to forget the serious affairs of life and to dawdle over story-books with- out aim or purpose. Was this efficient silent-reading drill, or was it largely a waste of time and money? Now the moral of all this is that neither a machine nor an educational method should be condemned as worthless because it fails in unpracticed hands. The successful working-out of any of the things mentioned above (espe- cially in the present state of our knowledge about them) demands energy, initiative, industry, enthusiasm, and a fairly complete understanding of the psychological princi- ples involved together with ability to apply them. The form of a method without its spirit will not ensure success INTRODUCTION 5 Before introducing a new mode of procedure into a school or school system, superintendents and teachers should be sure that they know what they are going to try to do, why they are going to try to do it, and what others have done along the same Hue. In addition they should have a definite plan as to how they expect to do it. This plan may have to be changed or modified several times before results are satisfactory, but nevertheless, some definite plan should be provided at the start. For information as to what others have done one must have recourse to educational books and periodicals. Much has been written, in one form or another, concerning supervised study, teaching children how to study, silent- reading, and standardized tests. Most of these writings, however, deal with generalities. Now, either through lack of inherent ability or through lack of proper training, it is undoubtedly hard for most people to apply general principles to specific cases; to take a suggested plan, think out the details, and put the plan into successful operation. This is the reason for such a large crop of failures when a superintendent calls his teachers together, tells them what he wants done, explains the general principles of the new plan, and leaves them to work out the details and produce results. The research men are giving us some splendid books as a result of their years of study and painstaking experiment. Their statements of procedure are, for the most part, sound in theory; and many of them have been proved by practice — especially when conditions were favorable. But these general principles, when applied under widely different and often adverse conditions, do not always work well. There are, however, some school people who are not easily discouraged, who take the trouble to make a careful study of the causes of failure, and who modify 6 STANDARDIZED TESTS either the plan or the local conditions or both ; and it is they who put the results of research into successful operation. Would it not, therefore, be well if we had more printed records direct from the actual workers in the field — from the men and women at the head of local schools and school systems and from their teachers? Even if these records were not of high technical or literary merit, they would tell just what teachers and superintendents have tried to do, and why and how they have tried to do it, together with the results achieved. The story might not always be one of shining success, but could it not be valuable to fellow workers for all that? If we are truly wise, we may often leam success from others' failures. Most of us profit by reading about the experiences of others in our own line of work, even though we are sometimes foolish enough to think we have nothing more to leam. Hence the excuse for this book. It tries to tell in a simple way how standardized tests and scales were used to improve the schools in a newly organized rural-school district in New Hampshire under conditions that were, to say the least, discouraging. In a way it is story. It is the story of how a corps of faithful, hard-working, but mostly untrained teachers, with the aid of an inexperienced superintendent, put standardized tests and measurements to practical use throughout a school system to the con- siderable advantage of all concerned; of how also, in con- nection with the use of the tests, they solved, with at least some satisfaction to themselves, the problems of efficient silent-reading drill, supervised study, and teach- ing children how to study, in one-teacher rural schools. It is not a story of complete and unqualified success; but I have endeavored to make it a faithful, readable, and understandable account of what was done, of why and how it was done, and of the results achieved. INTRODUCTION 7 The book is addressed to teachers and superintendents, both urban and rural. The principles of procedure set forth can be applied in a single school or a whole school district, in large schools or small, in city or country. The methods used have been tried out under about as unfavor- able conditions as could be found anywhere. Although the actual experiments have been carried on in a purely rural district, I can conceive of no reason why the same prindples cannot be applied with even greater facility, and the methods used to even greater advantage in town and city schools. Many of the obstacles that face the rural superintendent in attempting such a testing program as is here described are largely absent in the city. Better- trained teachers are available; the teaching force is more stable; and it is much easier to get the teachers together for frequent meetings. CHAPTER II THE PRACTICAL SITUATION^ A SUPERVISORY district containing twenty-six rural, iin- graded one-room schools with teachers for the most part inexperienced or untrained, a majority of whom had never worked under a superintendent before — this was the opportunity that fell to my lot when I became a New Hampshire superintendent under the new law effective September i, 1919. It was my first experience as a super- intendent. Truly the district was " virgin soil " as a member of the State Department remarked to me when I took the position. SiziNG-Tjp THE Situation The first two or three weeks were spent in traveling about the district getting acquainted with the teachers and sizing-up the situation. Most of the schools had already begim, the teachers having been hired by the local school boards as in the past. I found that three of the teachers were normal-school graduates, and that two had attended one summer school; but the others had no professional training whatever. Several had not attended school beyond the eighth grade, but had taught from ten to twenty-eight years in the same school. Three were in- experienced high-school graduates in their teens. Fifty per cent of all the teachers had never seen a professional book or magazine and did not know where to obtain one. All this, of course, had resulted in an ingrowing provin- cialism which could not but have a disastrously narrowing effect on their teaching. They were imitators of imitators. THE PRACTICAL SITUATION 9 Their methods were in imitation of the teachers who had taught them and who, in their turn, had imitated their own teachers. The results were the use of methods and texts so archaic as to be amusing if they had not been at the same time such a sad commentary on our boasted educational system. Most of the textbooks were sadly out of date. Arithme- tics, geographies, grammars, and even histories published in the eighties were in daily use. One of the local com- mittees insisted that they must be good books or they would not have worn so long. Of course the books were all based on the defunct doctrine of formal discipline and showed little of the psychological methods of presenting material which are the basis of modern educational pro- cedure. The idea seemed to be that an arithmetic is an arithmetic, and that a geography is a geography, one book being as good as another so long as nothing pertaining to the subject is omitted and the leaves are all present. I have emphasized the teaching and textbook situations because I consider them the most important factors. I shall mention a few other points briefly. As for the schoolhouses, it is sufficient to say that most of them were typical New England rural-school buildings of ancient vintage, modeled variously (according to the ideas of their instigators), upon churches, town halls, barns, and in some cases it would almost seem, upon pigpens. They were small, dirty, poorly heated, lighted, and ventilated, and in short generally unsanitary and ill-suited to their purpose. As to the organization of the schools, as I have already intimated, no attempt had ever been made to grade them. Another important factor in the situation was the attitude of the communities toward expert supervision. Three of the four towns in the district had always opposed such supervision as long as it was optional and were not in- lo STANDARDIZED TESTS clined to submit gracefully when, according to the pro- visions of the new law, it became compulsory. Such were the general conditions as revealed by my preliminary survey. ^ . However, there were three bright spots in the general darkness of the situation. First, the splendid new State school law which made the superintendent a State official with a pretty free hand and which provided liberal State aid for rural schools, where it might be necessary, to keep them up to the required minimum standards. Secondly, intelligent school boards willing to cooperate. And last, my own professional equipment which included a pretty thorough training in educational measurements and an earlier experience in the teaching of rural schools. Ac- cordingly, I tackled the proposition determined to show what scientific method in education, as I understood it, could do for rural schools if it had the chance. Analyzing the Situation The following week was spent in analyzing the conditions found to exist and in deciding upon the most efficient way to bring order out of chaos. As a result of this analysis a number of definite problems seemed to stand out clearly demanding early attention. From among these problems I shall select, for present discussion, only those, in the solution of which, as it seemed to me, standardized tests and scales could be used to good advantage. Problem i. To grade the schools fairly and accurately. Getting the schools graded in order to start the pupils right for their year's work was of course the most urgent problem. It would have been a comparatively simple matter to grade them arbitrarily on the basis of the teachers' judgments. But there are serious objections to this method even as a beginning, especially when the teacher THE PRACTICAL SITUATION ii is new to the school. If, for example, children are placed in grades lower than their parents think they ought to be, there are bound to be strenuous objections from some of these parents, and the only evidence one has to offer is that the teacher thought the children ought to be placed in those grades. Did you never hear this line of talk from an irate parent? " My John is just as smart as Mrs. Smith's Henry and he ought to be in the same class or above. It's a plain case of showing partiality. I Just won't stand it!" It is not by any means easy in such cases to prove to the satisfaction of the parent that the teacher had a sound basis for her judgment. And the fact is that she did not have a soimd basis. How did she know just what knowl- edge or ability a child should exhibit in order to belong in a certain grade? What standards of achievement did she have for the different grades besides her own arbitrary judgment? Giving ordinary tests for grading purposes is also in- accurate and unfair, because such tests are devised by the teacher or superintendent and hence represent merely the judgment of one of them as to what he or she thinks a child ought to know in order to be placed in a certain grade. Worse than all else, the children themselves are apt to become discouraged, not understanding wherein they have failed, and being at a loss as to just what they are expected to achieve. Even the superintendent and teachers, if they are conscientious, are not satisfied because they can- not feel sure that injustice has not been done. Retarding a child without good reason, thus robbing him of a year of his time, is a serious matter, at least for the child. What method of grading, then, could be found which would be fair to all concerned? After casting about for some time for a solution to this problem, it suddenly oc- 12 STANDARDIZED TESTS curred to me that standardized tests would help me out of my difficulty. Did they not offer definite standards of achievement for each grade? And those standards were not based on an unattainable one himdred per cent nor upon the opinions of dogmatic educators as to what a child ought to know at the end of a certain grade. They were experimentally derived and based upon the amount of work that normal children are actually doing in the differ- ent grades throughout the whole coimtry and not upon the amount of work somebody thinks they ought to do. By giving these tests in all my schools I thought I could determine just where each child belonged on the educa- tional ladder. The process would be roughly comparable to measuring a large number of sticks of various lengths by means of a tape measure sorting them into eight piles, each pile containing sticks of about the same length. Then, when angry parents wanted to know why their children were graded so low, I could show them. The children themselves could be made to see wherein they had fallen below standard and what they would have to do to come up to the standard. Both superintendent and teachers would feel that they were on solid groimd. Surely I needed standardized tests the first thing. Having reached this conclusion, I immediately ordered all the tests and scales I knew of that were sufficiently well standardized for my purpose. They covered the following subjects: arithmetic, silent and oral reading, spelling, writing, geography, history, and English language.^ Problem 2. How to measure the progress of pupils. Having settled upon how to do the preliminary grading, 1 For those who do not know standardized tests and want to get acqusunted, 1 recommend (or a beginning the purchase of Measuring the Results of Teacking by Monroe, and Educational Tests and Measurements by Monroe, De Voss, and Kelly. Both books describe the best tests and their uses and the former tells where to obtain them. These books are published by Houghton Mifflin Company. THE PRACTICAL SITUATION 13 the next problem was to find a just method of measuring the progress of pupils for promotion purposes. The same objections to teachers' tests hold good here as were noted in discussing the previous problem. This problem involved, obviously, the finding of a satisfactory system of marking. It has been proved be- yond doubt that ordinary teachers' marks are unfair, inaccurate, and generally unsatisfactory as a means of measuring progress or as a basis for promotion. In measuring the progress of pupils, standardized tests must surely find their widest field of usefulness. Here the process can be compared to standing a ten-foot measuring rod beside a young tree when it is a foot high and noting its growth in height. When it has added another twelve inches it belongs in the two-foot class. Twelve inches more and it is promoted to the three-foot class, and so on. It is entirely possible to devise a satisfactory system of marking, with the aid of such tests, as will be explained later on. Some of the teachers thought that the new State pro- gram of studies for elementary schools would be a sufficient guide for promotion purposes. This program is thoroughly practical and up-to-date. But it offers no definite, ob- jective standards of achievement from grade to grade. Therefore it is not a satisfactory tool with which to meas- ure the progress of pupils and to determine when they are ready for promotion. It is true that a printed program of studies specifies the subject-matter to be covered by each grade, but it is left entirely to the judgment of the teacher as to when that subject-matter is covered satis- factorily. The ground may be gone over more or less thoroughly and with very unequal results according to the methods and arbitrary standards of the individual teacher. Hence, it is obvious that programs of study 14 STANDARDIZED TESTS cannot take the place of standardized tests that are even fairly objective. Problem 3. How to measure the ability of teachers. One of the greatest needs of superintendents is a method of measuring the ability of teachers that will be accurate and fair to all concerned — a method that wiU not leave the superintendent open to the charge of favoritism or poor judgment — a method that will satisfy a teacher who has been rated something less than " excellent " that her shortcomings £ire real and not halludnations in the mind of the superintendent due to his personal dislikes and prejudices. What is wanted is concrete evidence of a teacher's ability or lack of ability that will allay carping criticism from whatever source. It is decidedly neither accurate nor fair to estimate a teacher's alnlity solely by observa- tion made by the superintendent during his visits to the classroom. It is not fair because (a) such observations do not furnish a sound basis for judgment; (&) the superin- tendent's opinions are quite apt to be colored by personal prejudices toward an individual teacher or her methods; (c) classes often show at their worst in the presence of viators; (d) even the teacher may fail to do herself justice under the critical eye of the superintendent. It is inac- curate for all the reasons noted above and because (a) some teachers do exceUent work when the saiperintendent is present and shirk all the rest of the time, and (6) if such teachers do their own testing, even the results may be made falsely to appear satisfactory. Provided a teacher is of good moral character with high ideals and a fairly pleasant personality, her further de- arability as a teacher is measured by the results she gets as determined by the progress of lie pupils when such progress is objectively measured. After all, it is results THE PRACTICAL SITUATION 15 we are after, primarily. Hence, the standardized tests measiire objectively both the progress of the pupils and the ability of the teacher at one and the same time. Problem 4. To find a practical method of supervising study and of teaching how to study. This may seem like a large contract for the one-teacher rural school with its crowded curriculum, but I shall try to show that it can be successfully worked out by using proper methods of study supplemented by judicious use of the standardized educa- tional tests and measurements. This chapter merely presents some of the problems con- fronting me in my new work together with some plans for their solution and reasons therefor. The following chapters will describe in detail the working-out of these plans and the results achieved. CHAPTER III GETTING TEACHERS TO FEEL THE NEED FOR STANDARDIZED TESTS Having concluded that standardized tests woiild greatly help me under the conditions set forth in the preceding chapter, I next attacked the problem of securing the whole- hearted cooperation of my teachers in their use. The psychology of interest teaches us that a person's best efforts are called forth only when he feels a real need. So I set myself to arouse in my teachers an enthusiastic interest by bringing them to feel a real need for definite standards of accomplishment in school work. In order to make them feel this need deeply, something more than merely telling them of the tests and their uses was neces- sary. They must be made to realize the inadequacy of ordinary methods of measuring the results of teaching. Accordingly, I called a series of teachers' meetings on four consecutive Satxirday afternoons. In a district more than twenty miles square it is not easy to get all the teachers together at one place. I succeeded, however, in persuading two automobile owners in each town to take the teachers to these four meetings at a price that would little more than pay for the gasoline consumed. Whether or not this scheme could be worked in other districts would depend, of course, on the character of the automobile owners and the success of the superintendent in convincing them that they ought to be willing to do it in an emergency and for the welfare of the schools. THE COOPERATION OF TEACHERS 17 The First Meeting At the first meeting I outlined briefly what I wanted to do and why, and then suggested that we try some practical experiments. The first experiment was intended to prove the inaccuracy of teachers' marks in general. A few days before, I had selected the paper turned in by a sixth-grade pupil in a regular arithmetic test given by one of the teachers. This test contained the usual ten problems and was supposed to measure knowledge of percentage. A hektographed copy of the pupil's paper was given out to each of the twenty-four teachers. Then, without warning them of my purpose or of what the results were apt to be, I asked them to correct the paper and mark it on a per- centage basis. When all had finished, the papers were collected and I at once tabulated the ratings on the black- board with the result shown in Table I. TABLE I. DISTRIBUTION OF THE RATINGS OF A SIXTH- GRADE ARITHMETIC PAPER Ratings Frequency 90-94 3 8S-89 8 80-84 6 7S-79 4 70-74 a 65-69 I Total S4 The effect was striking. Nearly everybody had caught the idea and the expressions on the various faces showed me that there was no need of rubbing it in by pointing out the moral. Aroused interest was evident as I passed the papers back with the suggestion that we analyze the methods of marking to discover the factors which pro- duced such evident lack of agreement. We finally agreed on the following points: i8 STANDARDIZED TESTS 1. Most of the teachers had ahned to mark the pupil's re- sponse to a problem entirely wrong if the answer was wrong, without regard to correctness of principle. 2. Some gave half credit if the principle was right and the answer wrong. 3. The majority marked answers entirely wrong if a decimal point was omitted or misplaced, while some gave vary- ing degrees of credit, if the digits of the answer were correct and the decimal point misplaced or omitted. 4. A few had assigned weights to the various problems on the basis of their own judgments with far from uniform results. This factor actually seemed to have been the one which had produced the most extreme variations in the marking. Then some one suggested that we try it again and see if we could not do better. Several others seconded the idea, so I passed around copies of an eighth-grade pupil's his- tory paper. Never did teachers work more seriously than did those teachers for the next fifteen or twenty minutes. The papers were then collected and the marks tabulated as before with the results shown in Table II. TABLE n. DISTRIBUTION OF THE RATINGS OF AN EIGHTH- GRADE HISTORY PAPER , Ratings Frequency 90-94 I 85-89 3 80-84 10 75-79 S 70-74 2 65-69 3 Total 24 Then followed a lively discussion, as a result of which the following important conclusions were unanimously agreed upon: THE COOPERATION OF TEACHERS 19 1. That teachers' marks are ordinarily very apt to be inaccu- rate. 2. That, due to the personal standards of the teacher and to individual marking systems, the work of the same child may be graded very high by one teacher and very low ' by another. 3. That such irresponsible rating may work serious injustice to the children. 4. That the work of different schools cannot be accurately compared under such conditions. And best of all, these conclusions were mostly arrived at by the teachers themselves with the help of a few lead- ing questions on my part. Was it worth while? Would those teachers ever again correct test papers with the same self-satisfied assurance of the infallible justice of their marks? Would they ever again feel fully Justified in retarding a pupil because he had been given a mark of 68 when the passing mark was 70? It was now time to bring the meeting to a close. Several wanted to know what better method of marking could be found. I advised all who were interested in solving the problem to order at once Monroe's book. Measuring the Results of Teaching,^ and to think the matter over until our next meeting. The Second Meeting In preparation for the next meeting I made a list of twelve history questions on the events leading up to the American Revolution. The questions were as follows: 1. How were laws made for the colonies? 2. What sort of governments had the colonies? 3. How was the commerce of the colonies regulated? I Monroe, Walter S., Measuring the Restdts of Teaching. Houghton Mifflin Company. 20 STANDARDIZED TESTS 4. What kind of a king was George ni? 5. What was the Stamp Act? 6. What were the objections to it? 7. What was the Stamp Act Congress and what did it do? 8. How did the British try to keep the colonists in order? 9. What friends had the colonists in Great Britain? 10. What was the Act of Association? 11. How was it enforced? 12. Distinguish between the real and the apparent reasons for the Revolution. When we were again assembled I passed two copies of this list to each teacher with the request that they rank the questions one, two, three, etc., in the order of their difficulty beginning with the easiest, and that they write these ranks opposite the questions on one of the papers which they had received. When they had finished, I asked each teacher to read the ranks she had assigned to the questions beginning with question one, while I tabu- lated them on the board in such a way that vertical col- umns would show the various ranks assigned to the same question and horizontal rows the ranks assigned to all the questions by each teacher. I regret that I have lost my copy of the original table, but the general arrangement (for four teachers only) is shown below without the correct figures. Teachers Rank assigned to each indicated question I 2 3 4 5 6 7 8 9 10 II 10 8 9 12 12 A 6 2 S I 4 7 12 S 2 II 8 12 7 3 3 4 9 9 10 6 12 10 4 9 3 4 7 II 8 6 I 7 S 12 6 8 I S II 2 II B C 2 D 3 THE COOPERATION OF TEACHERS 21 The tabulation, which I had made on the blackboard, showed such wide variations in the ranks assigned to the same question by different teachers, that a lively interest was aroused. In the ensuing discussion further conclu- sions were reached which may be simimed up as follows: 1 . That if a test is to measure accurately a pupil's knowledge of the ground covered, the questions must be so selected as to bring out the most important ideas in the subject-matter. 2. That there should be a large proportion of thought ques- tions in order to call for something more than a parrot- like repetition of memorized facts, so that the better students may be enabled to demonstrate their superior reasoning abilities. 3. That some accurate method of weighting the questions must be used if accurate measurement of pupils' abilities and knowledge is to be expected. 4. That the weighting of questions by individual teachers on the basis of their personal judgments is mere guess work, since, as was shown by our experiment, they cannot even judge accurately the relative difficulty of questions. 5. That the average of the judgments of a number of teachers is apt to be more accurate than the judgment of a single one. In line with this last conclusion we then proceeded to find the average rank assigned each question and, with these averages as a basis, to rank the questions according to their relative difficulty as determined by the combined judgments of the teachers. Next I asked them to use these ranks as a guide and to assign values to each ques- tion so that the sum of the values would equal one hun- dred. It was observed that for each question the varia- tions in values were not as wide as the variations in ranks had been. This was attributed to the fact that the order of difficulty used as a guide and the limit of one hundred 22 STANDARDIZED TESTS placed on the sum of the values would necessarily pennit much less variation. The average results are ^own in Table m. TABLE in. WEIGHTS ASSIGNED TO TWELVE HISTORY QUESTIONS ACCORDING TO TEACHERS' JUDGMENTS Average Average Question ' estimated estimated No. rank value 1 8 lo 2 4 4 3 9 " 4 I I S 7 9 6 S 7 7 2 2 8 6 8 9 3 3 lo II 14 - II lo 13 12 12 17 Total. ..100 The lesson of the day had been so well assimilated that some one suggested that since the ranking of the questions by each individual was all guesswork as proved by our experiment, then our averages were only averages of guesswork and hence likely to be far from reliable. For her part she did not see that we had any real proof that question twelve, for instance, was any harder than ques- tion three, and more than that she did n't see how we were going to prove that it was. Then came what I was hoping for, but hardly expected. "The eighth grades have recently completed their study of the period of history which tie questions cover. Why not give them as a test to these grades and so find out which questions are hardest? " This from a bright girl who had attended one siunmer session at Plymouth Normal. " How could you prove which questions are hardest by doing that? " I asked. THE COOPERATION OF TEACHERS 23 " Why, if the questions were given to a large number of pupils, would it not be safe to conclude that the one which is missed the most times is hardest and that the one missed the next largest number of times is the next hardest, and so on? " The practical side of this suggestion appealed to the majority of the teachers at once and they were eager to try it. Since this was the very thing I had intended to propose in order to get material for discussion at our next meeting, you may be sure that I was not slow in taking advantage of their enthusiasm. So it was decided to give this list of questions as a history test to all the eighth- grade pupils in the district. The teachers were to bring the corrected papers with them to the next meeting. The Thied Meeting I WATCHED the faces of the teachers as they gathered for our third meeting. The dazed look had passed from most of them by this time and a look of intelligent com- prehension was taking its place. The books which they had ordered at my request had arrived during the week and they had been reading them. They were apparently beginning to get their bearings. There was a business- like air in the way they took their seats and prepared for the afternoon's work that augured well for the future. The meeting having been called to order we began to investigate the results of the history test. It had been given to 106 pupils. As I called a question by number, each teacher told me how many times her pupils had missed it. I wrote the figures on the board, a column for each question. When the results had all been read they were summed and tabulated as in Table IV. This table showed that question four was the easiest since it was missed the least number of times and that *4 STANDARDIZED TESTS TABLE IV. RANK OF mSTORY QUESTIONS IN DIFFICULTy BASED ON NUMBER OF TIMES MISSED Question No — I 2 3 4 S 6 7 8 9 10 II 12 Times missed .. 28 44 40 8 23 20 3S 29 16 60 S6 SI Real rank S 9 8 I 4 3 7 6 2 12 11 10 question ten was the hardest since it was missed the greatest number of times; also, that question ten was seven and a half times as hard as question four because it was missed seven and a half times as often.' This would indicate that, in weighting the questions, number ten should be valued seven and a half times as much as num- ber four. When these points had been discussed, even the teacher who was dissatisfied at our last meeting was convinced that we had found a fairly accurate method of evaluating questions. The next step was to find values for the questions ac- cording to their relative difficulties. We first assigned to the easiest question (number four), which was missed eight times, a value of one. Since the next question in order of difficulty (number nine) was missed sixteen times, or just twice as often as mmiber four, we gave it a value of two. Question six was missed twenty times. As twenty is two and a half times eight, this question was valued two and a half. The same method was followed throughout. The sum of these values was found to be 51.5. Siuce the sum of the teachers' estimated values (Table HI) was one himdred, it was necessary for comparative purposes that the sum of these values should also be one hundred. Evidently if each of them were multiplied by two, the sum of the resulting proportionate values would be 103. 1 Statistical difficulties involving the location of the zero point were ignored. THE COOPERATION OF TEACHERS 25 This was near enough to one hundred for our purpose and it had the advantage of jdelding whole numbers. These may be seen in the last column of Table V, which sum- marizes the results of our study of the history questions. TABLE V. SUMMARY OF RANKS AND VALUES FOR EACH OF TWELVE HISTORY QUESTIONS Question Estimated Real Times Estimated Value from times The same multiplied by 2 No. rank rank missed value missed I 8 S 28 10 3-5 7 2 4 9 44 4 S-S 11 3 9 8 40 12 S-o 10 4 I I 8 I I.O 2 S 7 4 23 9 3-0 6 6 5 3 20 7 2.S S 7 2 7 3S 3 4.S 9 8 6 6 29 8 3.S 7 9 3 a 16 3 2,0 4 10 11 12 60 14 7-S IS II 10 II S6 13 7.0 14 13 12 10 SI 17 6.5 13 After the teachers had rated a few history papers using the two sets of values, we concluded our experiments. Crude and inaccurate they were, of course; but they were not intended to contribute to the statistics of educational measurements. They were rather designed to exemplify to the teachers that spirit of scientific investigation which is so rapidly making over our school systems, and in par- ticular to convince them that teachers' ratings as ordi- narily made are unreliable. And they had served well. All but two or three of the teachers had by this time expressed their firm conviction that tests devised by teachers and rated according to individual standards were of little use in finding out a pupil's real knowledge or ability, or in determining his standing with regard to other 26 STANDARDIZED TESTS pupils of his age and grade. These two or three teachers no amount of accumulated evidence could convince, be- cause everything was imalterably settled to their way of thinking before the experiments were tried. The old methods had been in use for hundreds of years and must, therefore, be better than anything new. It is useless to waste time with such people. The only thing to do is to get rid of them at the first opportunity. A few more points were cleared up by general discussion. Some one objected that, with a passing mark of seventy, it was evident from the scores that two thirds of the pupils would fail to pass the test. This brought up an important point. Is it fair to mark pupils on a percentage basis with loo standing for perfection? Some thought it was. Some thought it was n't. Most of them had n't thought about it at all, but since the matter was called to their attention they were inclined to think that it was hardly fair. What fairer way could I suggest? I then explained what is meant by a median score, demonstrated with figures on the board the difference between median and average, pointed out the advantages of the median, and then proposed that we find the median score of the io6 history papers using the derived values for marking. This was found to be 65.1, to which I added 10 per cent of itself and suggested that we use the result, 71.6, as a standard score. Any child who got this score would be given a mark of 100. That is, a pupil's mark would be the per cent that his score was of the standard score. For instance, if pupil A had a score of 26, his mark would be 26 divided by 71.6 or 36. If pupil B scored 63, his mark would be 63 divided by 71 .6 or 90. A pupil who scored 85 would receive a mark of 118. In using letters for marks this pupil would be marked A +. It was pretty well agreed that this method of marking would be emi- THE COOPERATION OF TEACHERS 27 nently fair, provided standard scores were available. The method may not be scientifically accurate; but it is cer- tainly fairer than the ordinary method of marking on a percentage basis, and it has the advantage of simplicity. Then some one wanted to know if we should do away with teachers' tests entirely and depend on the standard- ized tests altogether. We finally decided that teachers' tests should be used often by way of written reviews for the benefit of the pupils, but that the results should not have too much weight in determining the pupil's final standing. The pupils themselves, however, need not know how much or how little weight these tests might have on their grading for promotion. Most of the standardized tests which I had ordered were at hand for this meeting. Samples of these were given to the teachers to study in connection with their new books on educational measiirements. After explaining briefly the painstaking methods employed in deriving these tests and scales, I dismissed the meeting. The Fourth Meeting OtTR fourth meeting was devoted to the actual work of giving and scoring the tests. A few obvious facts were first emphasized, such as the need of accurate timing in cases where time was a factor, the fact that no help should be given the pupils other than clear and complete direc- tions, and the fact that when directions accompanied the tests they should be followed explicitly. I also warned the teachers to beware of copying on the part of pupils; and I then proceeded to administer the tests to the teach- ers just as they should be administered in the classroom. As each test was finished, we corrected and scored it, each teacher correcting and scoring her own paper for practice. In this way many obscure points were cleared up. Each b8 standardized TESTS teacher had seen each test given and had herself corrected and scored a sample of each test properly. They kept these corrected samples to use as models in correcting the tests which they were to give in their schools. Unless some such precaution is taken beforehand, the superin- tendent is sure to be surprised and dismayed when he gets the first packages of tests from his teachers. He will find that he must either return them to the teachers with time- consuming directions and explanations or go over most of them himself in order to secure accurate results. I had been through such experiences before on a small scale and intended to avoid them this time as far as possible. Since the tests were to be given to pupils the following week, each teacher then received a sufficient supply for her school. The tests were to be corrected and returned to me by mail within ten days. Unless a definite time limit is set and strictly adhered to, batches of tests from the various schools will come straggling in for two months after they are given out. Thus a few procrastinating teachers can delay the superintendent's part of the work to a very aimoying extent. In order to tabulate and study results for the district as a whole, he must have all his data in hand at one time and as soon as possible after the tests are given out. In this case I was particularly anxious to have all the tests in on time as the grading of the schools was being delayed pending the results. So I laid special emphasis on the ten-day limit. The teachers were further warned that, although I had no reason to distrust anybody, the matter was too im- portant to permit taking any chances. Accordingly, I proposed to check the work of each teacher by giving one or two of the tests in her school after she had given all of them. By comparing the results of my tests with theirs of the same kind, I could readily detect any gross careless- THE COOPERATION OF TEACHERS ag ness or intentional dishonesty on the part of the teachers. There is considerable temptation for some short-sighted teachers who know that their own efficiency is being measured by these tests, to stretch the time limit or to give illegitimate aid to the pupils, or even to drill on the test itself, in the effort to make their classes show up well. Of course any intelligent teacher, knowing how the tests are used, would see the short-sightedness of such a policy, since it is evident that what might be gained on one test would be lost on the next. However, two or three such dishonest teachers may, at first, work sad havoc with the accuracy of a superintendent's final figures regarding the efficiency of his schools and teachers unless he takes some precautions to discover the culprits at the start. Hence my method of checking their work. Above all, tests should not be permitted to fall into the hands of pupils. Since they are to be used again and again to measure progress, teachers should be impressed with the importance of safeguarding them. As fast as the papers are finished, they should be taken up to prevent possible copying of the questions or problems by the pupils in preparation for future tests. Of course, this does not apply in the case of the writing, spelling, and composi- tion scales because of the different manner in which they are used. In fact it is well to display these scales on the walls of the schoolroom so that the pupils may try to measure their own work. In this chapter I have attempted to give an account of how the cooperation of teachers was secured in the use of standardized tests. It may not be intensely interesting in the reading, but it was assuredly interesting in the doing. Results since have proved its value. I can recommend the general procediure to any one with a similar problem to solve. CHAPTER IV USING STANDARDIZED TESTS FOR GRADING PURPOSES In Chapter III I explained how the cooperation of teach- ers was secured in the use of standardized tests and how the teachers were instructed as to the manner of giving and scoring them. The chapter concluded with an ac- count of the distribution of the tests with directions that they were to be used at once in the schools. These schools, as I have indicated, were not graded. Indeed, it was the immediate purpose of this preliminary test to seciire a satisfactory basis for grading them. Supervising the Testing During the ten days allotted to the teachers for giving, scoring, and returning the tests, I visited as many schools as possible, giving advice and help where it appeared to be most needed. Considerable time and labor were thus saved in the later tabulation of results. Moreover, this plan enabled me afterwards to avoid much of the delay incident to the repetition of tests found on examination to have been carelessly or improperly given. Even so, it was not all smooth sailing. In spite of pre- liminary precautions to ensure accurate results in the shortest possible time, a number of tests had to be re- peated before I could feel sure that the results were fairly accurate. The returns of two or three teachers indicated such gross carelessness or incompetence or both that I was obliged to repeat the tests in their schools myself. But since the purpose of this Chapter is to show how the re- TESTS FOR GRADING PURPOSES 31 suits of these tests were used for grading purposes, I shall confine myself to that topic. Converting Scores into Grades The first step was to mark each paper with the grade corresponding to the score recorded on it by the teacher. For instance, one pupil's score on Woody's Multiplication Scale was 15. Since 15 is the standard score for grade vi, this paper was marked " 6." Another pupil's score on the same scale was 16. Since 15 is the standard score for grade vi and 17 is the standard score for grade vii, this paper was marked " 6 1/2." On Courtis's Silent Reading Test a certain pupil received the following scores: words read per minute, 140; questions answered in five minutes, 38; index of comprehension, 94. The standard scores for grades 11- vi are given as follows: VI Words per minute 84 113 145 168 191 Questions in five minutes 16 24 30 37 40 Lidex of comprehension .59 78 89 93 95 Hence the grades marked on the paper were, for words per minute, 4; for questions answered in five minutes, 5 1/3; for index of comprehension, 5 1/2. If teachers are instructed to keep all the tests of one kind together rather than all the tests of one pupil, this task of grading the papers is not interminable. As the teachers correct the papers, they should mark the score plainly on the front page. Then the superintendent may compare these scores with the standard scores and grade the papers correspondingly. About two days siifficed to mark the approximately two thousand papers from all the schools. The work was considerably facilitated by having on a II /// IV V 84 113 I4S 168 16 24 30 37 59 78 89 93 32 STANDARDIZED TESTS single sheet of cardboard the tables of standard scores for all the tests. Working at a large table with this sheet propped up in front of me, I could after a little practice ascertain at a glance the grade corresponding to any score. The advantage of such a procedure will soon become obvious to any one who attempts to mark a large number of papers with the standard scores scattered in a dozen diflferent books and pamphlets. (By way of an aside, I wonder why all the authors of tests do not print the standard scores on the front page. It would save a lot of time for the people who use the tests.) Devising a Geaph Card Next came the harder task of devising a method of re- cording res\ilts which would meet four conditions: (i) be in a form readily available for grading purposes; (2) be concrete and graphic enough to be clearly understood by teachers, pupils, and parents; (3) show on a single form small enough to be conveniently filed the standing of a pupil, a class, or a school in all subjects of the course of study for which standardized tests are available; (4) be in a form that could be used as the beginning of a continuous record to measure progress of pupils and ability of teachers. The class record sheets and graph sheets accompanying several of the tests were carefully studied with a view to adapting them to the purpose in hand. I soon decided, however, that they were too complicated and time-con- suming for practical use by the superintendent who must do all his own tabulating of results or have it done by un- trained teachers. Certainly such record and graph sheets would not meet any of the last three conditions mentioned above. They could not be easily understood by pupils and parents. There would be as many sheets as there were subjects; and several files instead of one would there- TESTS FOR GRADING PURPOSES 33 fore be required. Furthermore, such sheets do not ac- company all the tests, and only two or three tests make any provision for keeping permanent records of the scores of individual pupils. Some sort of simple graphic repre- sentation that would include all the tests on a single small sheet was absolutely necessary if the scheme was to be simple enough for practical use. After considerable experimenting, during which some of the graphs evolved resembled nothing so much as a lost trail in the desert, I finally adopted the following plan as both simple and practical. I ruled several 4X6 cards in copying ink with vertical and horizontal lines as shown in Figure i. The vertical lines were numbered at the top to represent subjects and phases of subjects in which tests had been given. The horizontal lines were numbered with Roman numerals to represent the eight grades of the ele- mentary school. From these originals nearly four hun- dred hektograph copies were made — enough for all the pupils in my schools above the first grade. The Meaning of the Graph Caiuj With these forms in hand and with the papers properly graded, it did not take very long to construct a graph for each individual pupil. Figure i is a copy of an actual record on file in my office. It is the graph of L. D., an eleven-year-old boy in the village school, Tamworth, New Hampshire. Table VI shows the same data includ- ing the subjects and phases of subjects corresponding to the numbers of the vertical lines of Figure i. L. D.'s grade for each test is shown in Table VI in the right-hand column opposite the name of the test. In constructing the graph a heavy dot was placed at the intersection of vertical line i with grade line iii to indicate third-grade ability in rate of silent reading; a second dot at the inter- 34 STANDARDIZED TESTS section of vertical line 2 with grade line vii to indicate seventh-grade ability in number of questions answered in five minutes; a third dot at the intersection of vertical line 3 with grade line in to indicate third-grade ability according to index of comprehension; and so on until the pupil's standing in all the subjects had been properly located by dots. The dots were then connected by a Nsme.LD. INDIVIDUAL RECORD Atre/I School ~71x.r>^wortf, \l;\\. qo. 2 3 4 5 6 7 8 9 10 II 12 13 14 15 16 17 isl YIII TU 71 T IV HI II I / \ 1 \ / \ \ ^ y \ / \ / \ \ / \ / N y / \ 1 \ / Fig. 1. Sample of iNorviDirAi. Recobd. Data the Same as ra Table VI heavy line giving the graph as shown. In actual practice the grades marked on the papers were translated directly into properly placed dots on the graph card without con- structing tables like Table VI. This can be done very rapidly with a little practice. Of course, for this purpose the papers should be sorted so as to get all the test papers of one child together. By this means each pupil's graph can be completed before taking up the next. The Older in which the subjects are entered on the graph TESTS FOR GRADING PURPOSES 35 No. I 3 3 4 S 6 7 8 9 lO II 12 13 14 IS i6 17 TABLE VI. RECORD OF L. D. (See also Figure i) Subjects and phases of subjects Rate of silent reading: Courtis's Test Writing speed Writing quality Questions answered in five minutes: Courtis's Test Index of comprehension: Courtis's Test Spelling Addition Subtraction Multiplication Division Mixed fundamentals Arithmetical reasoning: correct principle Arithmetical reasoning: correct answer Visual vocabulary English organization Geography History Grade III VII ni VII VII III IV V VI V IV IV VI V IVJ^ V card is significant. Reading is placed first because all grades are tested for reading ability. Spelling and writing follow, since they come early in school life. Then come the four fundamental operations of arithmetic, and so on, those subjects coming last which are taught only in the upper grades. Suppose the subjects were given in the following order: reading, history, spelling, arithmetical reasoning, addition, geography, etc. Then the graph of a third-grade pupil could not be drawn, since such a pupil would not be tested for history, geography, or arithmetical reasoning. The dots on the graph card indicating his grade of ability in the subjects which he does take would not be located on consecutive vertical lines. Hence no continuous line connecting the dots could be drawn. With the subjects arranged on the card in the order shown, 36 STANDARDIZED TESTS the graph will be complete as far as it goes, although it may extend only part-way across the card. Using numbers to represent the subjects on the graph card is not such a disadvantage as it might at first appear to be. One very soon learns to associate the subjects with their respective numbers. The slight disadvantage ex- perienced in the beginning is very much more than offset by the advantage of being able to use a much smaller card than would be possible if the names of all the subjects were written in. Grading the Pupils Next came the problem of deciding upon the grades in which pupils should be placed for the year's work. A glance at the graph for L. D. shows that he had an aver- age of about fifth-grade ability at the beginning of the school year. In fact the average of his grade as shown in Table VI is almost exactly five. Clearly, then, since the standards are Jime standards, he belongs in the sixth grade for the current year. Therefore the sixth-grade line on his card was emphasized by overlining to indicate that he is a sixth-grade pupil. Then, with his name, age, and the name of the school entered as shown in Figure i, the card was ready for filing. In the same manner a graph card was prepared for each child, his grade determined, and the card placed on file. Within two weeks from the time the first tests were re- turned, a graph card had been filed for every pupil (above the first grade) in the district, and each pupil had been assigned to the grade corresponding to the average ability indicated by his or her graph. A week later each teacher had received copies of the graphs of her own pupils in order that she might see where their weak points were and govern her work accordingly. TESTS FOR GRADING PURPOSES 37 Although in about a score of cases it was later found advisable to place a child a grade above or below that indicated by his graph, on the whole this method of grad- ing has proved surprisingly accurate. Most of the cases referred to were those of very bright children who, on the basis of the test results, would have been advanced to a grade very much above normal for their ages. The graph in Figure 2 illustrates a case in point. This eight-year-old INDIVIDUAL RECORD Name E.C Ago ?..... School ....C.hoSI>rU.a 2 3 4 5 6 7 8 9 lO 11 12 13 14 15 16 17 18 I VIII vu VI V IV UI II / \ / \ / N ^ \ 1 \ / ^ / \ / \ 1 1 Pig. 2. Sample op Individual Recobd of a Bbight Child (E. C.) girl shows an average of fifth-grade ability. Her mental age, as indicated by the Otis Group Intelligence Scale, is twelve years. She is fully capable of doing sixth-grade work this year. However, in spite of the opinions of some eminent educators to the contrary, I cannot convince myself that it is wise to rush such children through school, especially in a district like this where the elementary school is pretty sure to see the beginning and end of their school life. Is it not better to keep them in school until they are 38 STANDARDIZED TESTS at least fourteen, giving them a chance to do more and harder work than their classmates of mediocre ability, and supplying them with much carefully selected informa- tional supplementary reading to broaden their minds? There were four other cases of this kind in the same school. This, then, is a preliminary account of how standardized tests and scales were used to solve the grading problem. I believe that any one confronted with a similar problem will find the scheme practical. Nor will the labor involved prove either monotonous or iminteresting to one whose heart is in his work. CHAPTER V CONDITIONS REVEALED BY THE USE OF STANDARDIZED TESTS While making out the graph cards of individual pupils for grading purposes as described in chapter IV, I gradually became conscious of certain tendencies affecting the majority of the graphs. Not only was the variation great among individuals in different subjects, but there was a certain sameness in it that struck me as being significant of fundamental weaknesses in the school system. If the graphs had been on transparent cards and had been placed in a pile, their low and high points would have tended to coincide. That is, in certain subjects most of the pupils tended to grade high throughout the district, while in other subjects they tended to grade low. The Graph Cards Described In Figures 3 to 7, the numbers at the left of the horizontal lines represent the grades, while those at the top of the vertical lines stand for the various subjects as follows: ^ 1. Rate of silent reading 9. Writing, rate 2. Comprehension in reading 10. Writing, quality 3. Addition "• Arithmetical reasoning 4. Subtraction 12. English organization 5. Multiplication 13- Visual vocabulary 6. Division I4- Language 7. Mixed fundamentals of 15. Geography arithmetic 16. History 8. Spelling 1 This arrangement of subjects is somewhat different from the illustrative arrangement shown in chapter iv. 40 STANDARDIZED TESTS In each of these figures the heavy horizontal line is drawn at the grade in which the pupil was placed as a result of the September testing. The solid lines show the grading of the pupils according to each of the tests at the beginning of the year; the broken lines show their grading at the end of the year. In interpreting these figures it should be remembered that, as before stated, the grading FIGURE 3 SCQTT GRADE V CHICK'S CORNER SCHOOL a 3 4 5 6 7 8 9 lO 11 12 13 14 IS 16 8 7 6 5 4 3 2 1 A N ,/J \ \ 1 f ^^ --~, V""" V \\ 1 \ \ // /: \ / V // 1 1 \ / \ A f SEP EMS EH S :ORE \ / JUN 1 sc )RES ~ ■ FiQ. S. is based on June scores. In other words, if a child's score on any test equals the standard for the fourth grade, his performance is that of a child who has completed the fourth grade. Since his scores (obtained in September) indicate that he has already attained fourth-grade pro- ficiency, he will naturally be placed in the class which is ]ust beginning the fifth-grade work. In Figure 3 the solid line shows the curve of Scott, a twelve-year-old, fifth-grade boy in one of the Sandwich schools, at the beginning of the year. The high and low CONDITIONS REVEALED 41 points are more pronounced than in most cases. The solid lines in Figures 4 and 5 are the September curves of two other pupils drawn at random from among the cards of the Madison and Tamworth schools. All three of these pupils are normal or above according to the Otis Group Intelligence Scale. The broken lines in these figures are the end-of-the-year curves. FIGURE 4 EVERETT GRADE VII TAMWORTH VILLAGE SCHOOL 2 3 4 5 6 7 8 9 lO 11 12 13 14 15 16 8 7 6 S 4 3 2 1 / \ <^ J \\ / N ^ \ / / \l \ / K\ // \ ^ \, \ / \ / — SEP TEME ER S :ORE JUN E SC )RES PiQ. 4, It will help in the understanding of the figures to take the case of a particular child. Consider the record of Paul (Figure s). His score in the September test for rate of silent reading was 62. The third-grade score for rate of silent reading is 60. Accordingly, this boy showed third-grade abihty in rate of silent reading in the first test. The first point on the solid line is, therefore, located at the intersection of the third-grade hne with the vertical line I. In the June test Paul's score for rate of silent reading was 97. Since 97 is midway between 92 and 102, the 42 STANDARDIZED TESTS standard scores for the sixth and seventh grades respec- tively, the first point on the broken line is located about halfway between the sixth- and seventh-grade lines on the vertical line i. Similarly, in the first test his score for comprehension was 12, which is somewhat below the fourth-grade stand- ard. His score on the second tests, however, was quite equal to the standard of the seventh grade. FIGURES PAUL GRADE VI MADISON CORNER SCHOOL 2 3 4 5 6 7 8 9 lO It 12 13 14 15 16 e 7 6 5 4 3 2 1 > \ / \ ^- ■^^ _. .^ _^, ^ '/ Vn /' N, / . y \ ^ \ / / \l / > / k- ;'/ \ / \ / \ / SEP ■EMS ER S :ORE JUN E SC >RES Fio. 5. Tendencies shown by the Graphs The reader's attention is directed to the fact that in all the accompanying figures there is a tendency for the valleys and peaks of the curve for the June test to flatten out toward the horizontal. This represents the effect of the remedial measures taken in the interim between Sep- tember and June. It will be noted that these pupils stood high in the four CONDITIONS REVEALED 43 fundamental operations of arithmetic (3, 4, 5, 6, and 7) and in Greene's English Organization Test (12), which is mostly a test of general intelligence. In arithmetical reasoning or problem-solving (11), spelling (8), and lan- guage (14), they were near to grade standards. In rate (i) and comprehension (2) of silent reading, in rate (9) and quaUty (10) of handwriting, in visual vocabulary (13), and in the content subjects (15 and 16) each of these pupils ranked from low to very low. FIGURE 6 GRAPH SHOWING AVERAGE SCORES OF 62 NORMAL SEVENTH-GRADE PUPILS 1 2 3 4 5 6 7 8 9 lO 11 12 13 14 15 16 8 7 6 S 4 3 2 1 ^ Stf' ^^ •s ■"/ \ '^-., ■'? y "V X 'n N / / / / --'' V / \ \ ''A / \^ s, f \ / V ^ SEP "EMB ER S :oR^ s — JUN 1 SCI IRES Fig. 6. The graphs of 72.24 per cent of all the normal pupils above the third grade showed the same tendencies to a greater or less degree.^ Figure 6 is a sort of composite graph of all the normal seventh-grade pupils in the district. They numbered 62. Figure 7 is a similar graph for the 104 normal fourth-grade children. These graphs were ' In making this study the cards of all children who ranked below normal ac- cording to the intelligence tests were thrown out. 44 STANDARDIZED TESTS obtained by averaging the scores of the pupils in each separate subject and using the grades corresponding to the averages to locate the points on the graphs. For example, at the September test the average seventh-grade score for rate of silent reading was 95.2. This is a little above the standard fifth-grade score of 93. Hence the first point on the solid curve in Figure 6 is located just above the fifth- grade line. Similarly, the average of all the seventh- FIGURE 7 GRAPH SHOWINO AVERAGE SCORES OF I04 NORMAL FOURTH-GRADE PUPILS I 2 3 4 S 6 7 8 9 10 11 12 13 14 15 16 8 7 6 5 4 3 2 1 SEP ■EMB ER B :ORE JUN E SC )RES ^ X' ^■^^ ■'-; ;\ "/ ->^- N '~"^, S / / \' '" ■^ / / / \ / \ V ' \ 7 Pig. 7. grade scores in comprehension of reading was 17.8. Since this is about two-thirds of the way from the fourth-grade standard toward that of the fifth grade, the second point is located two-thirds of the way between the fourth- and fifth-grade lines. The Search for Causes; Over-Emphasis on Arithmetic Having observed in the graphs these general tendencies of which I have spoken, I next made a survey of the time- CONDITIONS REVEALED 45 tables and teaching methods in vogue in the district in order to formulate reasonable hj^otheses to account for the conditions thus revealed and in order to aid in planning remedial measures. The universally high scores in all the tested phases of arithmetic were indicative of over-emphasis on this sub- ject. The time-tables showed the relatively large amount of time assigned to arithmetic. The idea seems to be i prevalent among teachers, pupils, and parents that arith- metic is the all-important subject. These schools are no doubt t3^ical of the schools in small rural communities. Moreover, being widely scattered, they have little com- mimication with each other. It would seem, therefore, that the results of this study fairly represent conditions in most of the smaller rural schools where standardized tests have not been used and where their diagnostic values have not been realized. I am aware that these findings are contrary to those of more eminent workers who have foimd arithmetic to be a subject in which pupils usually grade low when measured by standardized tests. Possible reasons for this difference may be (i) that the results from all the pupils were con- sidered instead of only the normal ones and (2) that their tests were given in city schools. This over-emphasis on arithmetic in the smaller rural schools is not an unnatural condition. In fact there are sound reasons for it. In the first place, arithmetic is easier for the imtrained teacher to teach than the content subjects. Fair results can probably be obtained with less mental effort on the part of the teacher in the teaching of arithmetic than in the teaching of history and geography. The work is largely a matter of drill, and drill is easy for the teacher. In the second place, teachers usually have more immediately obvious success in the teaching of 46 STANDARDIZED TESTS arithmetic. It is human nature to like to do the things in which we best succeed. In the third place, pupils like arithmetic (a) because they enjoy good lively drill work in the fundamentals; (b) because they do not feel quite so much " at sea " in preparing an assignment of the next ten problems in arithmetic as they do in facing an assignment of the next ten pages in history or geography or physiology ; and (c) because children like puzzles. The older pupils especially enjoy solving the problems of this character which our textbooks abundantly supply. In the fourth place, when it is left to the children to divide their study time among the different school subjects — as has been the common custom in unsupervised rural schools — an undue amount of time will usually be spent on arithmetic. Not knowing what to do with improperly assigned lessons in the content subjects, but knowing that they must keep busy at something, children will turn to their arithmetics with which they feel most capable of doing independent work. Finally, the demands of parents on the teacher that their children be " learnt how to figger " is another factor in the situation. All these reasons cooperate in bringing about over-emphasis on arithmetic in the rural schools. As the beginning of an attempt to remedy this condition we decided to reduce for a while the time devoted to arithmetic by one half and to use the time thus gained for subjects in which the schools were making a poor showing. Other Tendencies Arithmetic was the only school subject in which there ap>- peared a general tendency to rank very much above grade. The only other markedly high spot in the graphs was that denoting their grading in Greene's English Organization Test. In this test the pupils demonstrate their ability or CONDITIONS REVEALED 47 lack of ability to rearrange broken sentences so as to make sense. Here is a sample of the disarranged sentences of which the test is composed: wanted, to go home, him, the dog Since this is largely a test of intelligence, the prevalent high scores of the pupils simply go to show that their low ratings in subjects other than arithmetic were not due to lack of mental ability, a fact also supported by their scores in the Otis Group Intelligence Scale. The average scores for problem-solving in arithmetic were close to grade standards for each grade. The reason why the pupils did not do as well in this particular phase of arithmetical ability as they did in fundamentals will be discussed a little later. Spelling and language are other subjects in which drill work figures very prominently. Although in each of these two subjects there were wide differences between the lowest and highest scores in each grade, the averages were well up to or above standard, as shown by the solid lines in Figures 6 and 7. The Writing Situation Writing averaged the lowest of all the subjects in every school but one. The teacher of this school had received business-college training and was good in muscular-move- ment penmanship. The low averages in writing led me to make a special investigation of the methods of teaching that subject in the district. A round of observation con- vinced me, not only that the teaching of writing was being neglected, but also that what teaching there was had little value. The copy-book method was in use in every school except the one just mentioned. The teachers in general did not know how to teach writing. Therefore they had little success with it and did not like to teach it. 48 STANDARDIZED TESTS Upon inquiry as to how the writing period was conducted, I learned that in several cases at least the teacher would simply tell the pupils to take their writing-books and write for ten minutes. During this time she would sit at her desk and correct papers. At the end of the period, without even looking at the copy-books, she would tell them to put away their writing materials and go on with other work. In very few of the writing periods that I observed personally was there any adequate attempt to teach the children how to write. Is it strange that the writing scores were disgracefully low? I wonder if this condition is typical of schools in smaller rural communities with untrained teachers, or is it a specialty in this district? 'i In an attempt to remedy the condition I tried to arouse the teachers to its seriousness, and I introduced a method of business writing into all the schools. This was some- what of a venture, since most of the teachers had had no training in muscular-movement writing. However, they were all informed regarding the correspondence course for teachers conducted by the publishers of the system, and were encouraged to take it. Several of them did so, and by the end of the year they were doing passable work as teachers of the new method. As a result of this radical change the writing conditions in the schools are now in a somewhat chaotic condition. It takes time to break up the old finger-movement habits and perfect new ones. Hence the graphs show little im- provement in writing scores for the year. We hope, how- ever, that during the coming year a continuation of our efforts will produce definite improvement in the quality of handwriting. The chief difficulty seems to be to get the children to use the muscular movement outside of the period of writ- ing drill. Hereafter, in order to further our efforts, no CONDITIONS REVEALED 49 written work will be accepted unless it is done with muscu- lar movement. We believe it is largely a waste of time to compel a pupil to write with muscular movement for ten minutes a day and then to let him use finger movement in writing his compositions. Poor Results est Reading, and Whv The reading scores were also scandalously low. Both the individual graphs (Figures 3, 4, and 5) and the grade graphs (Figures 6 and 7) exhibit this fact in a striking manner. Analysis of the situation furnished several quite probable reasons for the poor showing in silent reading. First, the pupils had not been taught silent reading. The reading drill in the schools was, and always had been, oral. Only two or three of the teachers had any concep- tion of what is meant by silent-reading drill. The oral reading was conducted in the old-fashioned way which needs no description — and mostly in a slipshod manner at that. The fact that the children were tested for silent reading when all their class work had been in oral reading was probably the chief reason for the low scores. Second, in most of the schools there was only one set of readers for each grade. The younger pupils knew most of the stories in the upper-grade books from hearing them read over and over by the older pupils. The fact that they knew the gist of these stories long before they ever reached the grades in which the books were used, that they had "studied " the lesson over several times at their seats (perhaps), and that each pupil was provided with a book in class, precluded any chance for real, live interest in the class work. Many of the teachers, even, did not seem to be over-enthusiastic. Third, the low scores resulting from the use of Thorn- dike's Visual Vocabulary Tests indicate that lack of word 50 STANDARDIZED TESTS knowledge probably accounted to a large degree for the poor results in reading. It may well be that the narrow range of reading, due to lack of variety in books and to the conspicuous absence of school libraries, was responsible for the limited reading vocabularies of the children. In the light of the above-described conditions the low scores in content subjects need little explanation. Success in history, geography, etc., depends on ability to study effectively. Efficient study is efficient silent reading. Even in arithmetic, much if not most of the difficulty en- countered by the pupils in solving problems lies in their inabOity to read and understand them as they appear in the text. Poor ability in silent reading, then, helps to explain why the scores in the problem-solving phase of arithmetic were so much lower than those in the fimda- mental operations. A New Policy as to Reading These matters were brought to the attention of the teachers. They readily concluded that reading is the most important subject in the school, because upon it depends success in most of the other subjects. We there- fore decided to give reading a place in the program com- mensurate with its importance. For the rest of the year most of the reading time was devoted to intensive drill in silent reading. Different methods of conducting this drill were devised in order to furnish variety and in order to keep interest alive. Much of the work in geography, history, physiology, civics, etc., was taken up as class drill in silent reading. Oral quizzes every few days by way of review in these subjects took the place of the customary daily question-and-answer recitation. In this way the time usually available for reading drill was quadrupled. Did so much reading drill get monotonous? The chil- CONDITIONS REVEALED 51 dren will testify that it did not. Did the content subjects suffer from giving up so much seat study and question- and-answer recitation? The graphs clearly indicate the answer. We adopted also a definite policy of vocabulary building. New words were constantly introduced to the pupUs by psychological methods. They were introduced as the names of ideas after the ideas themselves had been vividly brought to their attention by objects, pictures, or lively descriptions. The number of reading books in the schools was multi- plied by ten or twelve, and a generous beginning of school libraries was made. For the most part the new books were informational rather than merely entertaining. Yet they were books that appeal to children — and, indeed, they did appeal to them. Our difl&culty now is not in getting the pupils to read, but in getting them to do any- thing else but read. What did it all amount to? Well, look at the broken lines in the preceding figures. They speak for themselves. They are the graphs of the same pupils and of the same grades at the end of the school year. CHAPTER VI MEASURING THE PROGRESS OF PUPILS BY MEANS OF STANDARDIZED TESTS Teachers' Judgments oe Progress Unsatisfactory From time out of mind the estimate of a pupil's progress in his school work has been left to the more or less excellent judgment of his teacher, a judgment often warped by personal prejudice due to his behavior in school, his per- sonal appearance, or his father's standing in the commun- ity. The fact that the teacher gave tests and ranked the child on the quality of his reactions to them does not necessitate a modification of the above statement. For those tests were based solely on what sht judged the child ought to know concerning the various school subjects as a result of her particular line of instruction. She had no way of knowing definitely what a child of his age and grade really ought to know in order to be as well informed as other children of his age and grade in other schools. Even the grading of the papers, after they were corrected, was mostly a matter of judgment, as has been previously shown. Some of the more unthinking teachers took the testing and grading very seriously, marked the papers very care- fully on a percentage basis, and then " passed " the pupil or " flunked " him according to whether his mark was 70 or only 69. Others, realizing more or less vaguely the injustice of such a procedure, graded the pupils' work as excellent, good, fair, poor, or very poor, which they could probably have done just as accurately without giving any tests for grading purposes at all. THE PROGRESS OF PUPILS 53 Standardized Tests used to MEAsxntf: Progress But there is no longer any valid excuse for such hap- hazard methods of measuring the results of teaching in elementary schools. The standardized tests and scales furnish us with definite norms of achievement by means of which we can compare any child's work with the median or average for his age or grade and decide justly as to whether or not he is making normal progress. One of my purposes in using tests has been to measure the progress of pupils in their studies. Thus far ' we have given the tests four times in all the schools of the district. They have been given at intervals of several months so as to permit progress between tests to show plainly in the graphs. Three of these test periods — the ones particu- larly of interest in this chapter — fell within the school year 1919-20. All of the data from these several tests were graphically recorded and filed. The records are very interesting and highly satisfactory so far as proof of the efficiency of this method of measurement of progress is concerned, although, of course, they do not always show satisfactory progress on the part of the pupils. As heretofore stated, our plan is to give standardized tests in as many of the elementary-school subjects as possible to all the pupils in the district three times a year. They were given first in September, 1919, for grading purposes and to get a starting-point from which to meas- ure progress. In February, 1920, the tests were given again in order to find out how the pupils were progressing and particularly to discover along what lines, if any, unsatisfactory progress was being made, so that the teach- ers might see where increased effort or change of method • This chapter was written some time after the preceding chapters. 54 STANDARDIZED TESTS was needed. In June, 1920, they were given a third time s : s ., s ' (saaAv) ONIXiaM •ivno s K) s .. < 2 - s " s m a33dS o £ £ - ^ ^ . * « ° BNm3dS = C p 1 = = - t 1 0M1N08V3a BiSOaNOW •aoo c 2 <7. /' •NtHd ^ s V s \ Siva "^ :: " ) o N018IAia S 2 o / /\ " m t^ 'nnw CD J; s w / - *. ■xana " 2 /^ s * o " Nouiaav d 2 2 •l r ;; O" t O o i CE •dnoo 3OaN0H « ^ o. 31V u 2 2 S •a s snvav •dWOO S 1 s s :; s K 31V a •D s ? « 2 s 830VtJ0 g £ E - £ 5 = - CO 1 ► c - 1 < 5 - > DATE OF BIRTH AUG. 12, 1908 1 S 5 @a Sin o| ^ CO Is & - O H si •< In w S P h U [^ .si eo e o d (QCA THE PROGRESS OF PUPILS 59 third coincident with the fifth-grade line, and the second midway between and parallel to the others. Such a record would denote absolutely even and normal progress for the year. One of the tests given in the fall was the Cleveland Sur- vey Test in the fundamental operations of arithmetic- — ■ a test which is excellent for purposes of diagnosis. This test showed this particular pupil to be especially weak in the multiplication and division of fractions, decimals, and denominate numbers. Special corrective drill on these phases of arithmetic was responsible for the splendid progress shown on the multiplication and division lines. Note the very low score made in the mixed fundamentals test in September, the excellent progress made during the year, and the fact that in spite of such progress the pupil failed to come up to grade at the end of the year. It is noteworthy that only eight pupils in the whole district have so far succeeded in getting as high grades in this test as they averaged in the four fundamental operations, although the test is made up of a mixture of the identical examples used in the addition, subtraction, multiphcation, and division tests. Most of them fall below from half a grade to a whole grade. A study of Figures 9, 10, and 11 reveals the same facts concerning the results from this test. Although good progress is made in every case, the pupil or class persistently grades lower in this test than in the others on fundamentals of arithmetic. To my mind this indicates that the standard scores for this test are too high. Continuing the examination of Figure 8, we find Mon- roe's Reasoning Test in Arithmetic to be the next in order. This test is scored for three things: rate of solving prob- lems, solutions correct in principle, and correct answers. Good progress is shown for the year in all three although 6o STANDARDIZED TESTS the pupil fails to reach the grade standard for speed in solving problems. Spelling. In spelling ability the pupil accomplished twenty-five per cent more than a normal year's progress, with nearly four times as great progress made in the last half of the year as in the first half. And here is a chance for some more interesting comparisons of the graphs on the different cards. Figure 9 shows no progress in spelling in the first half; Figure 10 shows the same; while Figure 1 1, which is the record of a whole fifth grade, shows consider- ably more progress in the last half than in the first. The midyear tests revealed the fact that spelling work in general was progressing unsatisfactorily. As remedial measures, oral spelling drill, together with Buckwalter's Comprehensive Speller, was thrown into the discard. Ayres's Spelling Scale, supplemented by individual spell- ing lists made up of troublesome words from the pupils' own written vocabularies, was made the basis of the spell- ing course. A little booklet containing graded lists of 1600 " Common Blunder Words " was also used in most of the schools. Spelling lessons were shortened; new words were presented by a more psychological method; and the recitation consisted of a written lesson wherein the pupils use the words of the day's lesson in sentences or in a short composition. The efficacy of these changes in subject matter and method is strikingly evidenced by the greatly increased progress during the last half of the year. Handwriting. Next comes handwriting. This pupil's scores in writing (Figure 8) are typical of the general condi- tions revealed by the tests as discussed in chapter v; speed scores up to or much above grade and quality scores very low. Although this pupil showed considerable progress for the year, she failed to reach the grade standard in qual- ity of handwriting. But she did better than most of the THE PROGRESS OF PUPILS 6i pupils in this respect. Note that, throughout the year, her speed decreased while her quality increased. In the past, speed had been attained at the expense of quality. Now quality has been gained at the sacrifice of speed, and yet speed has not been reduced below the grade standard. Figure 9 also shows the fact that quality improved at the expense of speed. In most other cases, however, speed increased at approximately the same rate as quality, and the pupils were about as far behind in writing at the end of the year as they were at the beginning. All four of the records presented in this chapter show an improvement in handwriting for the year considerably above the average for the district. In general the improvement in writing ability was small. The reasons for the conditions found to exist at the beginning of the year and the general lack of progress during the year have already been discussed. English. As for language and grammar, so far as the author is aware, no satisfactory general test or scale has been standardized. One of our greatest needs at present in carrying out a complete testing program in the ele- mentary schools is a general language and grammar test somewhat on the same plan as the Hahn-Lackey Geogra- phy Scale. Starch's Punctuation Scale is good for measur- ing ability in that particular. Charters Diagnostic Lan- guage and Grammar Tests are excellent as far as they go, and they cover pretty well the common errors in the use of the English language. But no standards were available for them last year, so that they did not fit into a scheme which required tests that have been fairly well standard- ized.^ Hence we could do little in testing language and grammar ability last year. The two tests used, namely, Greene's EngHsh Organization Test and Thorndike's > Standard scores for these tests are now available and we are using them as a part of our testing program. 62 STANDARDIZED TESTS Visual Vocabulary Test might perhaps more properly be placed under the head of reading. The English Organiza- tion Test proved rather unsatisfactory. It does not seem to measure any definite ability. Its chief value seems to be in indicating, to some extent, a pupil's general intelK- gence or general reasoning ability, if there is such a thing, and even in this I have not found it to agree very weU with the results of regular intelligence tests. The vocabulary test, however, has proved very valuable, especiaDy in interpreting silent reading scores. There is a high degree of correlation between the scores in the vocab- ulary test and those of comprehension in silent reading if the scores of children much below normal are thrown out. When a normal child fails in comprehension of silent read- ing, an examination of his vocabulary scores will often show a serious lack of word knowledge, which can be reme- died by a definite plan of vocabulary building. To such a policy is due the excellent progress as regards vocabulary knowledge shown by the pupil represented in Figure 8. This progress is shown by the curves to be from fourth- grade ability in September to halfway between fifth- and sixth-grade ability in Jxme. Notice that this is also the highest point reached in the silent reading scores. This test likewise measures the efficiency of whatever method of vocabulary building may be adopted. Content subjects. Highly satisfactory in amount and uniformity was the progress in geography and history, as shown in Figures 8, 9, and 11, although for some reason the history scores persistently lagged behind those in geography. General progress. As before mentioned, Figure 8 is the record of a pupil a little above the average in intelligence and her record shows on the average, a little more than a normal year of progress ; which is as it should be. Further- THE PROGRESS OF PUPILS 63 more, her progress was in the direction of a more uniform ability in all subjects. The June curve is 35 per cent shorter than the September curve as shown in Figure 12 (a), thus approaching much nearer the ideal curve. This fact exemplifies the value of corrective measures based on diagnosis by standardized tests. These records are also used for promotion purposes. When a child's graph has moved upward over a space ap- proximately equal to the distance between two grade lines he is ready to be promoted to the next grade. As before stated, the pupil whose record is shown in Figure 8 was started on fifth-grade work at the beginning of the school year. Her graph has moved upward, as shown by the solid-line curve, until it averages better than fifth grade. This shows that she had attained fifth-grade end-of-the- year standards in June and was ready for promotion to the sixth grade and to begin work in that grade the following September. The Record of a Bright Child Figure 9 shows the record of a very bright eleven-year- old girl with a mental age of fifteen years and an I.Q. of 135. Although her graph showed an average of sixth- grade abilit}' at the beginning of the year, it was consid- ered wisest, because of her youth and because of various changes in the course of study, to have her take the regular sixth-grade work for that year and to prepare herself for double promotion by taking part of the seventh-grade work. Her chart shows a progress of from half a grade in rate of silent reading and spelling to two and a half grades in multiplication. In the June tests, as shown by the solid-line curve, she averaged halfway between seventh- and eighth-grade standards and was promoted to the eighth grade. Whatever of seventh-grade work she did 64 STANDARDIZED TESTS not take along with the sixth-grade work, she will take up in the eighth grade, thus losing nothing of subject-matter and gaining a whole year's time. Figure 12 (b) shows the > a. P X ^ - ~ "■ ~ ~ " " " ~ ■ ■ " " 1 NViaVH s « HOUViS " / ^. d 1 i A3X0V1-NHVH £ i = t Hoavxs " / £ , « 3 oc 6 z < / / / ■XONOd s.Hoavxs ^ > ■ 1* AavinavooA nvnsiA A --.. ^ ' DaO-0N3 9<3N33aO s = = = _ "_ »,. S s 2 (saaAv) ONIXtUM ■ivno s 4 ^ ' - S % n QildS F - Dj; !3? - s 5 s ? = :-. ? £ E = - I 3 X 1- £ < CNIN08V3U SiiOdMOH •aoo 2 / i, -•' .* •^ ■Niad s J n ;: aiva " ^ to .. * siviMawvaNnd aaxiw h / K / « <:«■ 2 > a NOISIAia " / ' " m •j-nnw 2 ^ ^ ■-. o / ^ *'. ■xans . =. . '• ■r: a " " Noixioav 1 2 ' e :: a 10 z o < 30UN0W •dWOO ) w = * * 3xva s » « 2 SHvav -dHOO 2 S s S 2 ^■ axva 5 ~ s S * n s % s3avao = F > > £ 3 = -1 Ul 1 p CO 9 < 1 e 0. 3 < IE __ >- DATE OF BIRTH AJLY 20, 1 BOS < c : 1 3 Si ^S SS S u Hcn li gh. s« W ?; ^■3 ^1 g^' U 6 S^ •^^ PS < ■< p ^H « i^S H^ w;^ & n « H ^1 "m sg ^ THE PROGRESS OF PUPILS 6S relative lengths of this pupil's September and June curves when straightened out. The June curve is about three fourths as long as the September curve. 1 1 NViaVH 1- « IS 2 Hoavxs mo f^ 1 A9XDVVNHVH Hoavis H M UD m ' J -lONOd S.HOaVi.8 C m to r- « ,• AavnnavooA ivnsiA c .--. — .--"—. -.^;;:;; 5 -oao-DNg S.3N33aO ____ _ _ ^.>.j-j^- (s3y^v) QNIXiaM ■Tvno ~~"; J g » _S-'»^S'""' -.-^ S S £ 5 -«:;^ , 0Nni3dS '^t's; ONINOSVBU SiBOaNOW ■aoo 3"-; » ; r. ".- 3j.vd ^- - :: » ^ ~ a ^^ i 8^vlM^wvaNnd c 3X1H -^--z TB ::: w .,":ii--'^ "" 1 ' " i '■ i i'---^ o o NOISIAia 22 5 ,. .™, ■xinw r-= — = — =—-;:=•-— _. •j.ans -r-r"= 2 - :.-• " NoiLiaav — = r-r--5---- = *^«;;...,' a z 5 3oaNow ■dwoo S — 7 o :i--r~o.^"C S O 0. 1» 1- « SHVav ■dWOO ^--- ^ « "" -;"' 2 3iva ,--^ -. 5 5 s g g S3avb0 II 3 p S > t s - "1 1 b. ul r. 16T MONTHS GRADE omoted to Grade UJ 1- g o < " m •- 2 , c a m ^ C H i M S o m 5 HO » a K M £i^ H S B H S 2 gl/3 E a Hi z P>3 ©■ d 66 STANDARDIZED TESTS The Record of a Dull Child FiGUEE lo gives the record of a very dull boy with a chronological age of thirteen years, a mental age of nine years ten months, and an I.Q. of 76. Note the great irregularity of the September curve and the general lack of progress throughout the year. Note that in many instances the scores of later tests fall below those of previ- ous ones, and that the reading scores are much lower than the vocabulary scores indicating that poor reading may be due to lack of native abihty and not to lack of word knowledge. This boy fell so far short of reaching fourth- grade standards in the June tests that he was not promoted to the fifth grade. He was already two years retarded. Question: Did we do right in retarding this child another year? Problem: What to do with cases of this kind in rural schools where special classes are out of the question, where manual trade schools are beyond the readi of the pupils, when promotion means placing the pupil wholly out of his depth, and when retardation means discourage- ment. This boy will probably never get beyond the fourth or fifth grade except through mistaken charity. Would it not be well to have some provision whereby such hopelessly retarded children could be permitted to leave school and engage in some useful and profitable work under the guidance of parents or other responsible persons, at least until society becomes sufiiciently civilized to make provision at public expense for the proper training of such individuals? They would at least be saved from forming habits of failure and idleness which so many such children acquire during years of forced attendance at school after they have reached the limits of their mental capacities in acquiring knowledge from books. Figure 12 (c) shows the relative lengths of this pupil's September and June curves. THE PROGRESS OF PUPILS 67 It should be remembered that all three of the pupils whose records we have been discussing were taught by the same teacher and no doubt in much the same way. >- s I ~ - T" p r-r p [— [- T~ r p p p NVIUVH s Hoavj.s 1 " ~ '" \ / 1 / /.3X0WNHVH 5 ; > ■\ ; Hoavxs f:! * v ; Q: 6 < / •j.0Nnd s.Hoavis » » "^ y' I .^ -* AuvnfiavooA ivnsiA '" '^ 1 / m ' ■OaO'ONB S.3N33aO 5 ? ■>. " V ■■■ *■" IS 2 3xva o 2 s s SWVQV •dWOO i I u> s m 2 p. 3iva « w s; s o s SBQVdD ggp^esa-i a UJ S 3 -o £ C!) m O UJ o i : a. 2 O O O M a I" 1^ O fig Winslow, Isaac Oscar, The United States, pp. gi fF. Boston: D. C. Heath and Company, 1910. READING IN THE UPPER GRADES 225 Section 3 Phoblem: Explain the formation of the Atlantic coastal plain. The strip of country between the Piedmont Belt and the ocean is generally low, level, and sandy. It is a part of the Atlantic coastal plain. All this land was once beneath the ocean. The soil was formed by the wearing effect of waves on the shore, or from mud carried into the ocean by rivers. 56 (234) Section 4 Question: How do the northern and southern coasts differ? There is a striking contrast between the northern and south- ern coasts. While in the North the sinking of the lands has produced islands, drowned valleys, and deep harbors, in the South the land has risen enough to make dry land of a strip of the ocean bed. Since mud was spread evenly by the waves over the ocean floor, this floor became level land after it rose above the water. The even bed of the ocean made an unbroken coast line, without many capes and bays. 86 (320) Section 5 Question: What are barrier reefs? Since this period of rising, the southern coast has settled again a very little. This has been an advantage by increasing the depth of water in the mouths of rivers and in the harbors. In the shallow water near the shore the waves have washed up sand from the bottom and formed long islands called sand bars or barrier reefs. In a similar manner a low coastal plain has been formed along the Gulf of Mexico. 75 (395) Section 6 Question: What are the Florida Keys? A large part of southern Florida is elevated but little above the sea, and portions of it consist of extensive marshes or 226 STANDARDIZED TESTS swamps. The largest of these is called the Everglades. The islands near the southern coast are called the Florida Keys from a Spanish word which means islands. 49 (444) Section 7 Question: How are coral reefs and islands formed? The surface of portions of the land in southern Florida and upon the Keys has been formed by minute animals called coral polyps. In parts of the ocean where the water is warm and shallow these animals live in great numbers attached to the bottom, and when they die their stony skeletons remain. They continually grow and die, and as the masses are raised above the surface, coral reefs or coral islands are formed, which the force of the waves slowly crumbles into soil. 84 (528) Section 8 Question: What three States of the Southern Section have no seacoast? In the Southern Section we may include the States bordering on the Atlantic Ocean and the Gulf of Mexico, from North Carolina to Texas, and the States of Tennessee, Arkansas, and Oklahoma. 32 (560) Section 9 Question: Why has the development of the resources of this section been so slow? These States, which are usually called the Southern States, are among the richest possessions of the country. For many years they have suffered the terrible results of the Civil War, in which they lost a vast amount of property and the Uves of many of their bravest young men. But since the close of that war they have advanced rapidly in wealth and general pros- perity. 65 (62s) READING IN THE UPPER GRADES 227 Section 10 Problem : Give some comparisons to illustrate the size of Texas. When the rich and well-watered lands of the "sunny South" are fully occupied and cultivated, they will provide homes for many millions of people. Texas is the largest of all our States. It is difficult to realize its size. It contains four times as much territory as the whole of New England. It is larger than any country of Europe except Russia. It is believed that it will at some time be able to support one half as many people as the whole coimtry now contains. . 87 (712) EXERCISE VII (Showing the adaptabihty of encyclopaedic material to effective silent-reading drill.) THE STORY OF COAL» Section i Ages before man lived upon the earth portions of it were covered with a dense growth of vegetation far more luxuriant than that found now in the densest tropical jungle. By the lowering of the level of the land these dense forests were cov- ered by the ocean, and while resting for ages on the bottom of the sea they were buried by mud. The land again rose and appeared above the ocean. The mud was hardened into irock and the buried vegetation by heat and pressure was turned into coal. This process was repeated many times through the uncounted ages and for this reason we find the coal in veins, one above another and separated from each other by layers of rock. Green plants can grow only in direct sunlight. Since the plants of the coal period owed their growth to the influence of the sun, and since the heat energy released by burning the coal is the same energy gathered from the sun's rays and stored in their ' O'Shea, M. V., editor, et d. World Book, vol. in, pp. 1442-43. Chicago: W. F. Quarrie and Company. 228 STANDARDIZED TESTS tissues ages ago by the plants from which the coal is made, coal is sometimes called buried sunshine, a very appropriate name. 191 Questions: 1. How did the dense forests of the coal period come to be covered with water? 2. What happened to them while thus buried? 3. Did the mud become rock under water or above? 4. By what means did the buried vegetation become coal? 5. Why is coal found in veins one above another? 6. What is the origin of the layers of rock separating the coal veins? 7. Why is BURIED SUNSHINE an appropriate name for coal? Section 2 Mineral coal, as hard and soft coal is generally called, dififers from charcoal in several particulars. Since it was formed under great pressure it is more compact, and since the air was practi- cally excluded during its formation many of the gases which are driven off in making charcoal were changed into sub- stances that combined with the coal. These are compounds of hydrogen and carbon with a few other substances and their presence in varying proportions gives us the different varieties of coal. 81 (272) Questions: 1. What name is applied to both hard and soft coal? 2. Why is coal more solid and compact than charcoal? 3. When coal is being formed, what becomes of the gases that cannot escape because of lack of air? 4. Why is air practically excluded during the process of coal formation? 5. What are the chief chemical elements composing these compounds? 6. What causes the difference in the different varieties of coal? READING IN THE UPPER GRADES 229 Section 3 Three general varieties of coal are recognized in commerce. The classification is founded on the degree of hardness and the varieties are anthracite, bituminous, and lignite. Anthracite is the hardest and best variety. It is often called stone coal because it is so hard and is supposed to have been the first coal formed, since it occurs deep in the earth. It was probably subjected to greater heat than bituminous coal, since it is almost pure carbon. The most extensive anthracite mines are in Eastern Pennsylvania. The veins do not lie horizontally, for they have been moved by mighty convulsions of the earth. Some are near the surface, while others are found at great depths. Anthracite burns with little or no flame and without smoke, and produces intense heat. Its chief uses are for warming dwellings and for manufacture of water-gas. 143 (41s) Questions: 1. Into what three general varieties is coal classified? 2. What is the basis of classification? 3. Why is anthracite often called stone coal? 4. What is supposed to be true of anthracite coal on account of its being found so deep in the earth? 5. Why is it beheved that anthracite was subjected to greater heat than bituminous coal in the process of forma- tion? 6. Which variety of coal contains the highest percentage of carbon? 7. What is Eastern Pennsylvania noted for? 8. If anthracite coal was formed at great depths, why is it sometimes found at or near the surface of the earth? 9. What are the chief uses of anthracite coal? I have devoted considerable space to the preceding silent-reading passages in an attempt to make clear to the reader the kind of material we have found best suited to class drill, and its arrangement in such a way that it can 230 STANDARDIZED TESTS be utilized by the teacher with the least amount of extra effort on her part. Ways of Using Silent-Reading Material There are several different ways in which such material can be handled so as to furnish live class drill. They may be summarized as follows: 1. The simplest way, and the one which demands least extra work on the teacher's part, is to give the class a cer- tain amount of time in which to read a definite amount of material and then call upon one of the class to tell, with books closed, as much as he can remember of what he has read. While he recites, the rest of the class watch for errors or omissions. When he has finished, other members of the class correct errors of statement and supply impor- tant details which he may have left out. This is good drill, but should not be used exclusively. There should be variety in silent-reading drill as well as in any other. The method possesses the special advantage of giving the child a large amount of practice in oral expression under condi- tions least conducive to self-consciousness. He really has something to say, some one to say it to, and an object in saying it. Ability to think and talk well at the same time develops with surprising rapidity under such condi- tions. 2. The class may be given a certain amoimt of material to be read in a limited time; and then, with books closed, the pupils may answer from memory questions asked by the teacher covering the important facts in the assignment. To save valuable time in class, and in order that the ques- tions may be well chosen, they should always be prepared and written down by the teacher beforehand. This is probably the poorest of the methods I am suggesting. It forces rapidity and concentration in the reading; but it READING IN THE UPPER GRADES 231 gives the advantage to the child with good memory, and it affords the pupils no definite objective in their reading. 3. The class may be asked questions, the answers to which are given or suggested in the material to be read. Then the pupils read to find the answers. The speed ele- ment takes care of itself here, since each child is anxious to be the first to find the answer. In order to give the slower readers a chance it is well to let the faster readers drop out of the game in turn as each answers a question correctly until a question is given to which none of the re- maining pupils can find the answer. Then one of the better readers may be permitted to answer it and the whole class will be in the game once more. To illustrate, here is a class of seven pupils: A B C D E F G Let A be the first one to discover the answer to the first question asked. He is permitted to answer it, and is thereby automatically debarred from answering the next one unless there is no one else who can answer it. Suppose that B gets the second question, and C the third, and that none of the four remaining pupils can get the answer to the fourth question within a reasonable length of time. The teacher may then appeal to A, B, and C, one of whom may answer the question correctly. Then she begins over again with the whole class. I have never yet seen a class fail to react with interest and enthusiasm to this type of drill. It gives the child a definite aim in his reading, and tends to develop the habit of rapidly skimming a para- graph or a page in search of a definite idea. 4. If the material is definitely arranged for silent-read- ing work with suitable questions printed before or after each section, or if the teacher's list of prepared questions is put on the board, the class may be asked to read the 232 STANDARDIZED TESTS questions for themselves as well as to read for the answer. This method has obvious advantages. It makes it abso- lutely necessary that the questions be prepared before- hand, and it makes the exercise more purely a reading drill. 5. A list of questions covering a certain topic or a cer- tain portion of the text may be put on the board, and the pupils may be given a limited amount of time in which to find and write out as many answers as possible at their seats. Such seat work is much more valuable than merely giving them a book to read at their seats with no time limit and no definite object in view. Right here let me call attention to the fact that the lists of " map questions " in geographies, especially in the older geographies, furnish an excellent type of sUent-reading material either for class drill or seat work. In fact, almost any sort of textbook containing lists of good questions based on the text will afford good material. These are a few of the ways in which we have varied the silent-reading work in our schools. Other, and perhaps better, ways will doubtless occur to bthers working along the same lines. Very few of the individual ideas incor- porated herein are original with us. Most of them have been used or suggested by other writers. We can claim only the credit of having gleaned them and put them to work extensively and systematically to further our aims. Any one of the methods listed above possesses this tremendous advantage over oral-reading class drill. Each individual pupil gets much more reading practice. In oral reading, only one pupil can read at a time, but in silent-reading drill when one pupil is reading they are all reading, and reading intensively with their attention con- centrated on their work. Hence, if there are ten pupils in the class, each pupil gets ten times as much reading practice. If there are twenty pupils in the class, each one READING IN THE UPPER GRADES 233 gets twenty times as much practice, and so on. And it is practice — especially practice with effort — that develops real reading ability. If any one believes that such changes in methods and materials as I have advocated herein will result in work that is dull and uninteresting to the child, he needs only to try them or to observe them in operation. He will then be convinced of the contrary, unless his mind is so hardened by prejudice that he can see no good in any new thing. CHAPTER XIV TEACHING CHILDREN HOW TO STUDY The title of this chapter may cause one to speculate as to the connection between standardized tests and teaching children how to study. The line of thought runs some- thing like this: the tests measure the results of study; results of study depend on efficiency of study; efficiency of study depends on the child's study habits as certainly as on his natural ability; and good study habits can be formed economically only by special training in the art of study. Hence helping children to improve their study habits is an important part of a systematic testing program designed to increase the efficiency of the schools. Children do not Know How to Study If there is any one point upon which everybody engaged in educational work can agree unanimously and unhesi- tatingly, it is that young people do not know how to study and that most of them do not learn how to study. Well, why should they know how to study or learn how to study? The ability to study efficiently is an acquired art. It can be acquired only through the practice of correct methods, and none too easily at that. Children are not born with the abihty to study and usually no one seems to take any very effectual pains to teach them. And yet, the child's chief occupation for five or six hours per day, five days per week, thirty or forty weeks per year, for from eight to sixteen years, is supposed to be study. Hundreds of thousands of men and women, at a cost of hundreds of millions of dollars per year, are supposed to TEACHING CHILDREN HOW TO STUDY J 235 be directing the study of the Nation's children. They assign lessons, order the children to study them, and have the children recite them. Still we hear the cry from teachers all along the hne in tones of despair: " Children do not know how to study! Children do not learn how to study! If children could but study efi&ciently, the teach- er's life might be worth living! " But what are teachers doing to help make their lives more worth living? How many of them are making any systematic, persistent efforts to help their pupils learn how to study to the best advantage? All too often the teachers themselves do not know how to study efficiently. Some of them, as I know from annoying experience, cannot even read well enough to translate accurately into action simple printed directions, such as those accompanying standard- ized tests. If the teachers do not know how to study, they cannot, of course, expect or be expected to have much success in trying to teach children the art of study. Children must be Taught to Study Nevertheless, if children are to study efficiently, they must be taught, or rather helped to learn, to study. There is no more reason to expect a child to become expert in the art of study without special training and directed practice than there is to expect a man to become an expert biologist or surgeon without special training. The fact that some few of superior mentahty who go on through high school or college become good students because of special aptitude and much practice is no argument that most pupils ought to be able to do likewise. The musical genius, without special training, can improvise delightful music; but most people need special training to make their musical efforts worth hearing. Measured by the amount of worth-while results obtained, assigning a lesson to the 236 STANDARDIZED TESTS average child with orders to learn it is about on a par with giving a person untrained in music a sheet of music with orders to play it. The results are usually discouraging. Poor Methods Prevail Poorly assigned lessons and the customary type of recita- tion have done much to encourage wrong methods, or rather wrong ideas, of study. Almost universally to the child, and only too often to the teacher, studying and memorizing have become synonjonous terms. The child, assigned the next five pages in history, and knowing from experience that he wiU be thoroughly questioned to test his remembrance of the details of the lesson or called upon to recite from a topic, knows no other resource than to memorize as many items as possible, or to memorize as much of the text verbatim as possible. Quite as often as not the teacher's questions cover so indiscriminately both important facts and inconsequential details that the child does not dare to neglect any detail even though he himself may have some decided ideas, and good ones too, as to what are the essential points in the lesson. The memo- rizing of facts to be repeated in class, or to be elicited in answer to suggestive questions on the part of the teacher, seems to be the dominant feature of so-called " study " in most schools. Many teachers, realizing the need for training children to study, have made serious attempts to meet the need. Some of them have succeeded in a measure, but more of them have failed through not attacking the problem from the proper angle. In many cases the teachers who failed were fairly good students themselves and were reasonably acquainted with the psychological principles involved in eflicient study. They took particular pains to provide an environment as favorable as possible to study. They told TEACHING CHILDREN HOW TO STUDY 237 their pupils to be interested in what they were studying, to concentrate their attention, to think about what they were reading, to study the relations of facts given in the book, and to associate the new knowledge with what they had learned before. All of this is perfectly sound advice and strictly to the point, but it is for the most part so much Greek to the children; and it would be difficult for them to profit by it through their own unaided efforts even if they could be made to understand it. What is the use of telling a child to interest himself in something entirely foreign to his natural inclinations when we as adults know perfectly well how practically impossible it is to force a real interest in anything that does not appeal to us? Of what use is it to tell a child that he must keep his attention absolutely fixed on what he is reading at his seat, whether he is interested in it or not, when he has such varied interests of his own outside of school toward which his mind naturally tends to wander ? Of what advantage is it to tell him to think about what he is reading when he has been given no definite problem to think about? What does the child know about making deliberate mental associations for the purpose of accumu- lating a store of orgam'zed knowledge? So, although the advice is good, it simply does n't " take." The teacher soon becomes discouraged and gives up trying, convinced that the children cannot be taught to study effectively. And she is right in her conviction. They cannot be taught how to study in the sense of being told or shown how. But they can be helped to learn to study. A child in the elementary school has no apperceptive basis that will enable him to understand or appreciate the importance of interest, attention, thinking, association, etc., in their relation to effective study, and he has not ordinarily the mental stamina of the superior adult to enable him to 238 STANDARDIZED TESTS apply them persistently of his own volition even if he were capable of understanding them. Something more is needed than merely telling a child how to study. He must be given assignments with very definite problems to work out and then be made to study these assignments under conditions that will force uncon- scious practice of the rules of effective study until proper methods of study become habitual. In other words, the child must be forced to study correctly if he is to learn to study through practice; and he must learn through prac- tice if he is to learn at all. But he will never learn to do the thing correctly by practicing it incorrectly and the latter is what most pupils are doing every day in our public schools. The brighter ones who attend school long enough eventually to learn to study ivith some degree of efficiency acquire their proficiency through the uneconomi- cal process of trial and error by means of which unsuccess- ful methods of study are recognized in time and dropped while more successful ones are slowly acquired. Situations which Favor Good Study Habits must be Provided Thus the child must be helped to learn how to study by being led to study properly. At the same time he is generally incapable of understanding and consciously applying persistently the psychological principles under- lying proper methods of study. Hence I say that the child must be obliged to apply these principles, not, of course, by the use of physical force, but through providing study conditions that will compel him to apply them unwittingly. This can be done without the principles being men- tioned as such or being discussed in any way. If the child has no immediate interest in the subject-matter to be studied, then the learning process itself must be made TEACHING CHILDREN HOW TO STUDY 239 interesting, or else the subject-matter must be made worth while from the child's point of view by connecting it up definitely with some problem in which he is vitally inter- ested. Concentration must be secured largely through furnishing frequent objects or goals during the study period. The habit of looking for fact relationships must be developed by thought questions which will compel the child to draw inferences from a collection of facts given in the text, and by supervised outline construction. The organization of knowledge through mental associations must be forced upon the child through actual practice in grouping ideas around a central thought and by obliging him to recall previous knowledge and experience in con- nection with every important idea in the lesson. AU this is possible of achievement as soon as the child has acquired independent reading ability, but it cannot be accomplished through ordinary methods of assignment, study, and reci' tation. Good Silent-Reading Ability Essential to Study Elsewhere in this volume it has been stated that effi- cient study is efficient silent reading. This is entirely true only if we conceive of efficient silent reading as imply- ing not only thorough comprehension of words and sen- tences, but also the weighing of thoughts, the evaluation of facts, the classification of new ideas presented by the text, and their association with previously accumulated knowledge. If efficient silent reading is held to be merely the comprehension of the author's thoughts as expressed in the text, then the statement needs to be modified some- what. But even with this less inclusive conception of what constitutes efficient silent reading, it cannot be de- nied that good silent-reading ability is the basis of all efficient study from books. Rapid silent reading with 240 STANDARDIZED TESTS comprehension is the most fimdamental factor in eflScient study. The child cannot be greatly interested in words and sentences that he does not imderstand. He cannot concentrate with profit on material that has httle or no meaning for him. He cannot study the relationships be- tween ideas unless those ideas are comprehended. It is past dispute that a child must know how to read well before he can study well,- and he must know how to study before he can produce better results to be measured by standardized tests. Herein lies the connection between this chapter and aJl that goes before. ' Good Silent-Reading Methods Encoubage Good Study Habits In addition to the fact that good silent-reading ability is essential to effective study, I submit that properly directed silent-reading drills furnish the most practical means available for developing good study habits in children through actual practice in the factors of efficient study, namely, interest, attention, thinking, and the correlation and association of ideas. I have said that children must be forced to practice these things by means of study con- ditions that will compel such practice. Silent-reading drill furnishes the medium through which the forcing can be accomplished. The mental processes involved in effective drill of this sort are ahnost exactly the same as in efficient study. The types of silent-reading class drill heretofore de- scribed have, with us, proved very effective in holding the child's interest and forcing concentration of attention on the matter being read. Probably more often than other- wise in the content subjects the children have httle inter- est in the subject-matter itself. But they do thoroughly enjoy lively silent-reading class drill. Their interest is TEACHING CHILDREN HOW TO STUDY 241 mostly in the immediate objects to be achieved, namely, to see who can read the paragraph or page in the shortest time and understand it well enough to give the principal facts of the assignment from memory and in a connected manner; to see who can be the first to discover, from among the details of the text, the answer to a definite question; or to see who can first discover the main thought of a paragraph. The chief interest is in the spirit of lively competition engendered by these drills; but, nevertheless, the interested, active, enthusiastic cooperation of the pupils is secured under conditions that make for real im- provement in ability to study. It is hard for any one who has never tried these methods to realize the degree to which they force children to con- centrate their attention on the work in hand. With the timed-section method it is manifestly impossible for the pupU to reproduce completely and coherently the main thoughts of the section read unless he has read it atten- tively; and, since no one but the teacher knows who is going to be called upon to recite after the reading, every member of the class must read attentively. With the question method, even though most of the questions are answered directly in the text, the chUd must read atten- tively and understandingly if he is not to miss the answers when he comes to them. If the answer to a question is merely suggested by a fact mentioned in the text, so much greater is the demand on the attention of the reader and real thinking is introduced as an element in the reading. If the answer to a question must be inferred from several related facts given in the text, then the highest type of reading abiHty is demanded in that the child must not only concentrate his attention on what he is reading, but he must do real thinking in associating various ideas with each other in their proper relations and with the main 242 STANDARDIZED TESTS idea expressed in the question. Furthermore, he must hold the main thought in mind aJl the time he is doing the reading in order not to miss pertinent facts expressed in the text. Some Examples Let me illustrate some of these points. Here is a section the answers to which can be taken directly from the text. The sugar maple, otherwise known as the hard maple, is one of the very best of our shade trees. It is well shaped, affords dense shade during summer, and in autumn becomes beautiful because of the rich and varied colors of its foliage. In the spring its blossoms unfold with the leaves. It grows more slowly than the soft maples, but it requires less moisture and is more useful and durable. Questions: 1. Why is the sugar maple a beautiful tree in autumn? 2. Which comes first, the blossoms or the leaves? 3. Which grows faster, the hard or soft maple? These three questions are given in order of difficulty, or of their thought-producing power. The first answer can be taken directly from the text; " because of the rich and varied colors of its foliage." The second question requires understanding of the fact that, in the sugar maple, the leaves and flowers appear at the same time. The third one goes a step farther and requires that the pronoun " it " be associated with the maple tree, its antecedent, and that the terms " sugar maple " and " hard maple " be asso- ciated in the mind as representing the same thing. Here is another paragraph with a question that requires still more thought on the part of the reader. Maple sugar, like that made from sugar cane, is darker than ordinary brown sugar, unless the impurities are removed. To TEACHING CHILDREN HOW TO STUDY 243 do this, milk or beaten eggs are stirred into the boiling sap. This causes most of the coloring matter to rise to the top and mingle with the froth, which is then skimmed off. Those who live near sugar bushes enjoy making and eating maple wax. This is formed by letting the hot sirup fall on snow or ice. Question: How may light-colored maple sugar be obtained? It is evident that finding the correct answer to this question necessitates some real thinking and association of ideas on the part of the pupil. He must connect up the ideas that maple sugar is ordinarily dark in color, that the dark color is due to colored impurities, and that these im- purities can be gotten rid of in a certain way. The following paragraph and questions demand even greater concentration and still more extensive thinking on the part of the pupil if he is to answer them correctly. The leaves are the food-making organs of the plant, and the sugar that is made from maple sap in the spring was made the simimer before by the leaves of the tree. As the sugar is manufactured it passes down from the leaves into the trunk and roots of the tree, and is stored in the living cells of these parts in the form of starch. Then when food is needed in the spring to enable the buds to grow and expand into blossoms and leaves, and to produce the seeds, the starch is changed back to sugar, which is dissolved out of the storage cells and carried upward in the sap. Questions: Is there much sugar in the roots of the maple tree in winter? Why? All the data required for inferring a correct answer to these questions are given in the paragraph. But in order to obtain the correct answer, the reader must comprehend several facts in their proper relation, namely, that during the summer the leaves of the maple tree manufacture more food in the form of sugar than is needed by the tree for immediate use; that this surplus food is carried to the 244 STANDARDIZED TESTS roots where it is stored for use the following spring; and that the soluble sugar is changed to insoluble starch as it is stored in the living cells of the roots, remaining there as starch until the following spring when the sap begins to rise. Hence in winter there is little if any sugar in the roots of the tree. The understanding and correlating of these facts so that the correct answer may be inferred calls for much practice in real thinking. Habits thus Developed Carry over into Other Work In this way good silent-reading drill compels practice in proper methods of study. Such practice tends to develop good study habits which may logically be expected to carry over into such a closely related activity as seat study. That they do carry over, and thus improve the pupil's ability to study by himself even poorly assigned lessons in the content subjects, is evidenced by the num- ber of teachers who have spoken to me regarding the greatly improved study ability of the majority of the children after one or two years of intensive drills of the type described in the chapters on silent reading. Other proof lies in the fact that the pupils' test scores in the content subjects, very low in the first tests, are increasing at much more than the normal rate, wMle, at the same time, no extra time or special drills have been given in these subjects other than connecting them up with the silent-reading word. Other types of silent-reading drill, to be described hereafter, serve to stimulate still more strongly the higher thought processes and are thus still more conducive to the development of good study habits. Study should also be Directed or Supervised A FAIR proportion of school time devoted to silent-reading class drills, in which the teacher employs the methods and TEACHING CHILDREN HOW TO STUDY 245 kinds of material used in the better schools of this district, and described and illustrated in the two preceding chap- ters, together with properly assigned lessons in the con- tent subjects, will do much toward solving the problem of helping children to learn to study to good advantage. But such procedure does not include all that can be done to improve study habits of children or to economize study time. In order to get the best results possible, the forced practice in proper methods of study must be on the actual lessons assigned on the content subjects. Furthermore, if we are to make sure that the conditions under which the studying is done are those most conducive to economy of time in learning, and the development of good study habits in the children, the study must be di- rected. This brings us to my next topic, namely, " Super- vised Study." Because of the obviously close relation between proper supervised study and " teaching how to study," several important things that might be included in this chapter are reserved for fuller discussion in the next. CHAPTER XV SUPERVISED STUDY Possibly the greatest source of waste in the public-school system is the time supposed to be spent by the pupils at their seats or at home in trying to " study " their lessons, by their own unaided efforts. Not infrequently half the pupil's school time is spent in " studying " from books to prepare for the recitations. Three Types of Activity in " Studying " Some pupils deliberately idle away their study time be- cause they have no interest whatever in the school or its work. Compelled by law to attend until they are fourteen, or sixteen, or eighteen, as the case may be, they are sullen and obstinate and plan only to kill time until they are old enough to leave school. They are usually of inferior mental ability and the source of most of the serious dis- ciplinary problems. Quite often their attitude toward the school is the result of retardation and consequent dis- couragement due to the narrowness of a program of school activities which provides no suitable and interesting work for the moron type. Many other pupils read over their lessons in a dilatory, ineffective manner with their minds constantly wandering to more inviting fields of thought in their own sphere of interests. This is largely because the lessons as assigned present no definite problems in which they are interested. This group usually contains bright and energetic boys and girls who need only to have their school work enlivened and their tasks made definite in order to become inter- SUPERVISED STUDY 247 ested and enthusiastic workers. There are still others who do not waste their time in idleness and day-dreaming, but who are busy every minute of their study time. Yet the time and energy even of these pupils are largely wasted because of ineffective methods of study and ill- directed efforts. They are industrious, and of good mentality; and they would really like to study if they but knew how. As it is they merely do the best they can under unfavorable circumstances. Difficulties in Connection with Sxipervised Study A GROWING realization among educators of the tremen- dous waste connected with undirected study has led to the development within recent years of various plans for supervised study in the more progressive school sys- tems. The idea of supervised study is good, and it promises much in the way of increased efficiency when it is properly carried out; but, as supervised study is ordi- narily conceived, the practical difficulties in the way of its general adoption are prohibitive in the smaller school sys- tems and particularly in the one-teacher rural schools. The main difficulty is for the teacher to find the time both to conduct the recitations and to supervise the study periods. The ideal solution, as it is most generally advo- cated, is to have two teachers in each room, one to conduct the recitations, and the other to supervise the study of the children at their seats. This would necessitate a doubling of the teaching force that would be financially out of the question in most, if not all, school systems. Personally I doubt whether this solution would be ideal, even if it were financially practicable. The teacher who supervised the seat study could do little more than see that every child kept busy at his lessons and assist individuals over difficulties. Often her efforts would be duplicated by 248 STANDARDIZED TESTS having to assist several different pupils over the same difficulty separately. Any attempt to explain or illustrate difficult points to the class as a whole would be apt to interfere with the recitation supposedly going on at the same time in the same room. Of course, keeping the children busy and giving individual assistance to the slower pupils would undoubtedly be beneficial; but the benefits to be derived from such practice would scarcely justify the great additional expense involved. In school systems where each teacher has to teach only one undivided grade or one division of a grade, the problem of introducing supervised study may be solved with com- parative ease and with no additional expense. Under such circumstances the teacher is, for the most part, free during the study periods to direct the studies of the pupils as may seem advisable. But here again, if her efforts are limited to keeping the children busy and to assisting indi- vidual pupils, the full possibilities of the study period are far from being reahzed. What is needed is directed class study so organized as to develop in the pupils methods of attack that will constantly improve their ability to do inde- pendent work. Merely keeping them busy and assisting them over the rough places will not do this. Another device, and a more practical one, is the divided period, one part of the period being devoted to supervised study, and the other to the recitation. The recitation may follow the study immediately, or tlie last half of the period may be given over to study in preparation for the next day's recitation so as to give the pupils a chance to do further study outside of class if they so wish. But even this scheme is available only in high schools where the periods are long, or in schools where the teacher has only one or two grades so that the class period and the study period for each subject can be combined into one fairly SUPERVISED STUDY 249 long period. In the smaller school systems, where most or all of the teachers have charge of from four to eight grades, the time of the teacher is necessarily all taken up with recitations and the class periods are so short that division of them is not feasible. In each of these class periods there is little more than time for a hasty testing of the pupil's preparation of the lesson and for the assignment of the new lesson. Another difficulty hes in the fact that so-called super- vised study, in the hands of teachers who do not rightly understand its purposes and possibilities, or who do not care to put themselves to any more trouble than is neces- sary to enable them to draw so much per week in salary, may degenerate into a procedure more harmful than bene- ficial. The chief features of such a procedure have been mentioned above, namely, keeping the children busy and giving a certain type of assistance to individuals. Keep- ing the children busy at their lessons probably can do no harm, but the wrong kind of assistance can do much harm. It is quite possible for the direction and help offered by the teacher to tend to make the child dependent and utterly unable to do a piece of work for himself. In other words, the teacher may make herself a crutch rather than a guide. The result of this kind of assistance is that children do not learn to work independently. Hence their inability in the upper grades, in high school, and even in college to use their study time to good advantage. The giving of assistance at the right time, in the right way, and in proper amount demands a type of judgment based on a sense of values that is all too rare in teachers. If supervised study is to be nothing more than giving undiscriminating as- sistance to individual pupils in specific difficulties — and with an unskillful, or careless teacher, it is apt to be little more — we shall probably do better without it. 350 STANDARDIZED TESTS Supervised study is closely connected with teaching children how to study in that it furnishes the best oppor- tunity for such teaching. Teaching how to study does not involve a systematic course in psychology. It con- sists in guiding pupils in the actual practice of the art, giving them suflftcient directed practice in it to make cer- tain reactions habitual, and then making them conscious of the best methods of study by calling attention to the elements in their study experience that have meant de- cided success or failure. Any supervised-study plan that does not result in constantly increasing the ability of the pupUs to do independent work is largely a failure. Here as often the correct point of view is that a teacher's chief aim should be to make herself as imnecessary to her charges as possible and as soon as possible. Hence we must help children to learn how to study in connection with their actual studying. In order to do this effectively their study must be properly directed, or supervised. , Silent-Reading Drill Offers an Opportonity for StiPERviSED Study Now, I do not propose to submit any complete plan for supervised study that is applicable imder any or all condi- tions. I do not hope to offer a complete solution to a problem that has long been puzzling the minds of much abler men. But I do hope to offer some suggestions that will prove helpful in attempting to solve the problem in actual practice. With this in view I am about to describe some procedures now in vogue in the best schools of my district which appear to help materially along these lines. These practices may or may not be correctly termed " supervised study," but since from them seem to be de- rived most of the benefits generally ascribed to supervised study, they may well serve as substitutes for the latter, SUPERVISED STUDY 251 whatever they may be called. They have two advantages at least: (i) they are applicable so far as I can see in any school or school system however small, and (2) some of them can be handled with profit by any teacher with average intelligence and teaching abiHty who is willing to give them an honest trial. I have mentioned in a former chapter how, in our zeal to increase the reading ability of the pupils quickly as the first step in increasing the efl&ciency of the schools, we decided to amplify the time devoted to reading drill by taking up some of the content subjects, notably physiology and hygiene, and civics, by the silent-reading methods hereto- fore described. This was by way of experiment and was undertaken at first with considerable misgivings, especially on the part of the teachers. It was feared that the children would not " get " the subjects if they were not studied at their seats and recited in the orthodox fashion. In order to test the efiicacy of these methods in fixing the principal facts of the subject-matter in the minds of the pupils, weekly oral or written quizzes were given to- gether with monthly reviews for which the children pre- pared by reading over at their seats the ground covered by reading and discussion in class during the month. The results were highly gratif3dng from the first and soon re- moved all doubts as to the effectiveness of the procedure. And, from the psychological viewpoint, why should it not be effective? The strict concentration of attention com- pelled by these silent-reading drills is much more favorable to retention of subject-matter than is the usual perfunc- tory reading at seat in so-called study. Moreover, the reproduction of the principal thoughts and the class dis- cussion of important points help the pupils to discriminate, consciously or otherwise, regarding the things they ought to remember. 252 STANDARDIZED TESTS In fact, so effective did this method prove in producing results and improving learning ability that, by the middle of the first year, several of the best teachers of the district were taking up much of the regular geography and history work as silent-reading class drill supplemented by frequent quizzes by way of review. As explained before, practically all of the supplementary historical and geographical read- ing was taken up in this way in the regular reading classes. After giving the tests several times and noting the progress made by the pupils in the content subjects in schools using this method, it occurred to me that these silent-reading class drills were proving themselves to be really efficient forms of supervised study. It also seemed clear that with supervised study definitely in view the exercises could be so improved as to develop in the children the ability to think and to organize the facts gleaned from their reading, thus improving still more rapidly their ability to study independently. In short,, properly planned silent-reading drills gave promise not only of de- veloping the ability to read rapidly and comprehendingly (which is the very foundation of study ability), but also of stimulating the higher thought processes and of giving actual practice in the art of efficient study. This idea was presented to the teachers and I found that the same thought had occurred to some of them as a result of their experience with the silent-reading work. Several of them made excellent suggestions and offered to cooper- ate in developing the idea. It is some of the processes evolved from this beginning that I am about to describe. The simplest form of silent-reading class drill that can be said to have much real value is the timed-section method which has already been explained in detail. This was the method we used first in all the silent-reading work. It forces concentration of attention and helps greatly in the SUPERVISED STUDY 253 accumulation of factual knowledge; but it does not compel much thinking or organization of ideas, and hence is not specially valuable as a means of putting supervised study into effect. Class discussions of the important points in the lessons in their relations to the main topics will, how- ever, help to overcome this defect. The Question Method is Especially Advantageous Although the timed-section method has some advantages peculiarly its own and is still much used in reading supple- mentary material in story form, in the better schools it was soon superseded for supervised study purposes by the question method also described in a former chapter. It remains in general use only by the weaker and less inter- ested teachers of the district. Indeed, it appears to be the only method of silent-reading popular or practicable with that class of teachers since it demands less extra work and less skill in handling. With the question method, however, the questions can be so selected as to call attention to the main points in a lesson and their relations to each other and to the main topic. The pupils may thus be led in spite of themselves to do some real thinking in addition to having the principal facts fixed in their minds and getting the best kind of reading practice. Let us see how content material can be handled by the question method of silent-reading drill so as to get reading practice, information, and practice in the art of study at the same time. Suppose that the topic to be studied is " The Birch Tree," in Moseley's Trees, Stars, and Birds, and that the text assignment is chapter iv, entitled " Birches." The teacher should prepare beforehand a list of questions covering the main points in the lesson. A good share of them should be real thought questions to which the answers can be inferred only by reading so as to 254 STANDARDIZED TESTS grasp the relationship among the facts given in the text. Most of them should be formed so that they cannot be answered by words or phrases taken directly from the book and without association of ideas by the reader. Of course, when the object is merely to call the attention of the class to some detail whose significance is derived from its bearing upon some thought question to be asked later, a question answered directly by the text is permissible. It would be advisable at first for the teacher inexperienced in the use of this method to make an outline of each topic to be studied and then to frame her questions to cover all the subtopics in the outUne. The Method Illustrated The following selection is a part of the chapter under con- sideration divided into sections for silent-reading drill and with suitable questions following each section. BIRCHES Section I Because of their grace and beauty birches are a favorite subject for landscape artists and photographers, and they are frequently planted in parks and on lawns. The white and the paper birch are the species of birch most frequently planted for ornamental purposes. They are especially effective when placed among evergreens, because of the contrast in colors. Many of the white birches have slender, drooping branchlets with deeply cut leaves that might be taken for those of some varieties of maples. White birch grows wild in Europe and Canada and to some extent in our Northern States, but with us the paper birch is more common. Where a forest of spruce or of certain species of pine — as white pine — has been burned, paper birch and aspens spring up. In the abundant sunlight ' From Moseley's Trees, Stars, and Birds. Copyright, 1919, by World Book Company, Yonkers-on-Hudson, New York. SUPERVISED STUDY 255 of the open spaces these trees grow more rapidly than seedlings of the spruce or pine, and a forest of birch and aspen grows up in place of the evergreen forest. Questions: 1. Why are birches so frequently planted? 2. What are the two favorite birches for ornamental pur- poses? 3. Why are they often planted among evergreens? 4. Where does white birch grow wild? 5. Which is the more common in the United States, the white birch or the paper birch? 6. When a forest of spruce or white pine is burned what kinds of trees grow up to take their place? 7. Which grows more rapidly, birches or pines? Section II The yellow birch has yellowish or silvery-gray bark which has an aromatic odor. The bark of the white birch and paper birch is creamy or pinkish white and splits into paperlike layers. From birch bark the Indians made canoes, as well as boxes, buckets, baskets, kettles, and dishes. In making their canoes they stitched together large plates of birch bark with the fibrous roots of white spruce, coating the seams with resin obtained from spruce and pine trees. In parts of northern Europe the bark of the white birch is used for shingles. Boats made from it are used on the Volga River. From it are made birch oil and birch tar. Russia leather has an aromatic odor due to the oil of birch bark used in tanning it. As the odor repels insects, this leather is valuable for binding books. A few such bindings in a bookcase are a safeguard against insect enemies, and this oil is said also to protect books from mildew. Questions: 1. How could you tell the bark of yellow birch from that of wliite birch if you were blindfolded? 2. What color is the bark of the paper birch? ase STANDARDIZED TESTS 3. What is said in this section that would indicate the origin of the name "paper birch"? 4. What uses did the Indians make of birch bark? 5. Describe how the Indians made canoes of birch bark. 6. What is the bark of the wliite burch used for in parts of northern Europe? 7. Name four other uses of birch bark in Europe. 8. Why is Russia leather especially valuable for binding books? Section III The wood of the white birch is used as a fuel for smoking hams and herrings, because of the flavor which it imparts. Being Ught colored, soft, and easily worked, it is used for mak- ing spoons, ladles, bowls, and fish casks. Spools, wooden shoes, ox yokes, chairs, and tables also are made from it. Charcoal made from it is burned in forges, and soot made from birch fires is used for making printer's ink. The wood of the paper birch is used for fuel, shoe pegs, spools and toys. The yellow birch and sweet birch yield wood that makes fine fur- niture and a good interior finish for houses. It is often stained dark red and varnished. It is then said to have a "mahogany finish." Few trees are useful for so many purposes as is the birch. Questions: 1. Why is white birch wood used as a fuel for smoking meat and fish? 2. Name several things made from white birch wood? 3. What part does the white birch play in the manufacture of printer's ink? 4. Name some things made from the wood of the paper birch. 5. What kinds of birch are used for furniture and finishing lumber? 6. It is often finished in imitation of what costly wood? 7. How does birch wood compare in variety of uses with that of other trees? SUPERVISED STUDY 257 Section IV Have you ever thought of any connection between the size of a tree's leaves and the coarseness or slenderness of its branches? Even in winter, birch trees look quite different from ash or hickory, not merely in color but in the appearance of the branchlets. The function of the branches is to hold the leaves up to the light, and the number of branches required depends on the size of the leaves. Trees with small leaves, like birch, elm, and wUlow, have very numerous branchlets. Those with large leaves, like ash and hickory, do not require so many branchlets. The leaves themselves reach out to the light and fill up the spaces in the crown of the tree. Most palm trees, of which there are a thousand kinds in the tropics, do not branch at all, but they have immense leaves with long stalks to reach out to the light. Questions: 1. Of what use to a tree are its branches? 2. Upon what depends the number of branches which a tree needs to have? 3. Why do such trees as the birch and ehn need so many branchlets? 4. Which has more branchlets, a hickory or an elm? Why? 5. Seeing a tree in winter, when it is bare of leaves, how could you tell whether its leaves in summer are large or small? 6. Why is it necessary for most palm trees to have very large leaves with long stalks? Section V On the twigs or small branches of a tree look for small oblong and elevated places on the bark. These are called lenticels. They are breathing pores through which the air can enter to reach the living inner portion of the bark and from which water vapor escapes. On birch and cherry trees the lenticels may be seen not only on the branches but even on the trunks. Here they have become elongated by the growth of the bark. 258 STANDARDIZED TESTS Lenticels are to be found on all trees. Where the bark is very thick, as it is on old oaks, they are at the bottom of deep cracks. Questions: 1. What are lenticels? 2. Where would you look for them? 3. What are they for? 4. What is their shape? 5. On the trunks of what trees may they be seen? 6. What trees do not have lenticels? 7. Why can they not be seen on the trunk of an old elm tree? Let us consider the first section in detail with a view to discovering how well the questions cover the facts in the text. Most of the important ideas in the section are in- cluded in the following list: 1. Birch trees are graceful and beautiful. 2. They are often planted for ornamental purposes. 3. The white and paper birches are the favorites for planting purposes. 4. Their white color is particularly beautiful in contrast with the evergreens. 5. The paper birch is the more common of the two species in this country. 6. The destruction of a forest of spruce or white pine by fire is followed by a growth of paper birches and aspens. 7. Birches are fast growing trees. Now compare this list with the questions following Section I. Are there any ideas in this list that are not brought out in the questions? Does the list omit any im- portant ideas given in the text? Does the list contain any ideas not pertaining to the main topic? Is there any doubt but that the reading of these questions one at a time and searching out the answers will impress these facts upon the child's mind more surely and permanently than would the mere reading of the text at seat? If you doubt it, try it. SUPERVISED STUDY 259 I met with striking evidence on this point during the first year that the method was used in our schools. One day in the fall I demonstrated the method in a certain school for the benefit of the teacher. The reading mate- rial was part of the chapter on Petrograd in Carpenter's Europe, for which I had a hst of questions prepared. This school had a new teacher for the spring term and it became necessary to demonstrate again. I had kept the list of questions in my notebook for demonstration purposes; and I used them again in this same school believing that the children had forgotten all about what they had read hurriedly and but once several months before. To my surprise, several of the children scarcely glanced at the book as I asked the questions in order, but were ready with the answers about as soon as I had finished the ques- tions. Fully half of the questions were answered in this way. If I ever had any doubt as to whether material read in this way would " stick," it disappeared then and there. The questions pertaining to Section I are not real thought questions. Not every paragraph furnishes data upon which to base such questions. This is one that does not. However, if the reader will examine the other sec- tions in connection with their accompanying questions, he will find a number of questions that demand real thinking if they are to be answered correctly. In fact, the answers to more than half the questions carmot be picked directly from a single word or phrase in the text. Alternating Supervised Study and Recitation With questions prepared beforehand, this amount of text can easily be covered in ten minutes. Let us suppose that ten minutes is all the time available for this class. At the end of the period the pupils are told to be prepared for a review of this lesson on the following day. The next day 26o ■ STANDARDIZED TESTS they are called upon to answer the same questions with their books closed. Discussion of interesting points is encouraged as long as it does not wander too far from the main topic. On the third day another topic is taken up in the text as on the first day, and on the fourth day this topic is reviewed from memory. This process may con- tinue for ten days, and at the end of this time a general review may be given either as an oral or written quiz. The questions for this review are selected from the original lists which the teacher keeps in her notebook. In this way the reading drill and supervised-study period comes on one day and the recitation on the next. If twenty minutes are available for the class period (or even fifteen if the class is mostly composed of good readers), the period may well be divided even though the lessons have to be shortened. In this case the first half of the period should be devoted to review of the preceding lesson (the recitation) and the last half to the reading of a new lesson (supervised study). Thus the children may have opportunity before the recitation for further study if they so desire. Few of them, however, will find further study necessary. With the divided period, the general reviews may come of tener — say at the end of each week. It is not well to have these reviews cover too much ground; and it is surprising how much ground can be covered in a week with a fifteen or twenty minute divided period. This is especially true if the children are fairly good readers to begin with, and if they as well as the teacher have become accustomed to the method. One need have no fear that it will be impossible to cover the ground required, if part of the class period, or even every alternate period, is devoted to supervised study of this kind. Fully as much ground can be covered; and, judging from our experience at least, the results will be SUPERVISED STUDY 261 quite equal to or even better than those usually obtained by the traditional daily question and answer recitation following a period of unsupervised seat study. With only fifteen minute periods some of our classes cover three or four different texts in one subject in the course of a year, thus obtaining a much broader view of the subject than they could possibly have if they were confined to the study of a single text. Another advantage is, that if most of the content sub- jects are handled in this way, ordinary seat study in these subjects becomes less and less necessary for most of the pupils. Accordingly, the time usually spent in seat study can be used for extended supplementary reading or in any other way that may seem advisable. One way of using this time to good advantage is for the teacher to put questions on the board covering assignments in textbooks and to have the children find the answers and write them. Children generally like to do this sort of work. The questions give them something definite to work on at their seat. This type of work, however, should never be al- lowed to supplant the regular silent-reading class drill, because it takes no account of the speed element in reading nor does it force concentration to anything like the extent that class drill does. Some textbooks contain at the end of chapters excellent lists of questions which may be used for this purpose. Often, however, such lists of questions are not suitable; and the teacher should go over them carefully before assigning them for this type of seat work. Judging from some of these lists, one would almost think that the person who framed the questions had never seen the book. After trying a few such lists the pupils are apt to become discouraged and lose interest — and one cannot blame them. I believe that this method of handling silent-reading 262 STANDARDIZED TESTS drill for purposes of supervised study will prove, on the whole, to be one of the most useful that can be devised. It demands less extra work, and less skill on the part of the teacher; and it covers ground more rapidly than most of the other methods. At the same time it produces excellent results from every point of view. On the one hand, it can be used to good advantage by the teacher who is somewhat lacking in originahty or initiative after she has had it ex- plained and seen it demonstrated; but on the other hand it furnishes plenty of scope for the abler teacher to display whatever skill she may possess. The more skillfully it is handled, of course, with fuU understanding of its purposes and possibiUties, the more efficient will it prove in develop- ing abUity to study independently and the better medium it will be for practice in the art of study. A great deal depends upon the skill and judgment used in framing the questions. Another Way of Conducting Supervised Study — , Finding the Topic of a Paragraph There are other ways of conducting silent-reading drill, however, ways that make greater demands on thinking abihty and that do more to help pupils acquire the ability to grasp relationships among ideas. The ability to or- ganize ideas in their proper relations to each other and to some central idea or group of ideas is an abihty that de- velops only through actual practice. Pupils must be led to do such organizing if they are to learn through practice. These other methods are specially valuable as aids in developing organizing ability in pupils; but they consume much more time and demand much more skill on the part of the teacher if they are to prove successful. Neverthe- less, they should be used whenever the requisite time and skill are available, for they help to develop in the child an ability that is vital to his success as a student. SUPERVISED STUDY 263 One good tj^e of such silent-reading drill for supervised- study purposes consists in having children read paragraphs from a text which has no paragraph headings or topic sentences, then requiring them to suggest suitable head- ings or topic sentences for the paragraphs. Get sugges- tions from as many different pupils as possible for each paragraph and write them all on the board. Then call on the class to decide which one is the best and why. If at first the children seem at a loss to know what is required of them and no suggestions as to suitable paragraph head- ings are forthcoming, or if from a number of suggested headings no satisfactory decision as to which is the best one can be reached, do not tell them and do not be dis- couraged, but lead them to analyze the paragraph under discussion. Have them consider each sentence or state- ment in the paragraph with a view to discovering a com- mon or central idea to which each statement or sentence refers. Let me illustrate. A short article selected at random from an encyclopsedia is on Cocoa. Let us proceed to analyze the second paragraph which reads as follows: The cocoa is a small tropical tree cultivated extensively in Ecuador, Venezuela, Brazil, Saint Thomas Island (off the West African coast), Ceylon, the West Indies, and Central America. Heat, moisture, and a deep rich soil are the conditions which favor its growth. The straight, regular trunk usually attains a height of twenty feet, and puts forth branches which bear shin- ing, oval leaves, dark green above and red underneath. The flowers, which have five narrow, bright-red petals, grow directly from the trunk or from the older branches, and are almost stemless. The fruit, a cucumber-shaped pod with a thick, deeply-grooved rind, has the same pecuharity. Each pod con- tains many almond-like seeds, covered by a thin, reddish-brown shell, and within each of the seeds is a dark brown kernel, the 204 STANDARDIZED TESTS valuable portion of the plant. The seeds have the commercial name of cocoa hearts, while the kernels are called nuts. Sentence i. "The cocoa is a small tropical tree cultivated extensively in Ecuador, Venezuela, Brazil, Saint Thomas Island, Ceylon, the West Indies, and Central America." This sentence tells us where the cocoa tree is cultivated. Sentence 2. "Heat, moisture and a deep rich soil are the conditions favorable to its growth." Growth of what? The cocoa tree. Sentence 3. "The straight, regular trunk usually attains a height of twenty feet, and puts forth branches which bear shining, oval leaves, dark green above and red underneath." This sentence speaks of trunk, branches, and leaves. Of what? The cocoa tree. Sentence 4. " The flowers, which have five narrow, bright- red petals, grow directly from the trunk or from the older branches, and are almost stemless." This sentence teUs of the flowers and where they grow. Flowers of what? The cocoa tree. Sentence 5. "The fruit, a cucumber-shaped pod with a thick, deeply-grooved rind, has the same peculiarity." This sentence described the fruit. Of what? The cocoa tree. Sentence 6. "Each pod contains many almond-like seeds, covered by a thin, reddish-brown shell, and within each of the seeds is a dark brown kernel, the valuable part of the plant." This sentence describes the contents of the pod or fruit. Of what? The cocoa tree. In this way it is made evident to every member of the class that each sentence in the paragraph has reference to a single central topic, the cocoa tree. The central idea of the paragraph, tersely expressed as a topic, is a paragraph heading; or, briefly expressed as a complete statement, it is a topic sentence. This is, of course, slow work for the children at first, but it is well worth while. Most of the children will grasp the idea by the time five or six paragraphs have been analyzed, SUPERVISED STUDY 265 and after that it is merely a matter of practice. In a sur- prisingly short time they will acquire the ability to recog- nize the topics of paragraphs merely by reading them through carefully and thoughtfully, providing, of course, that the material studied is well organized. In selecting material for such drill, the teacher should be very careful to see that it is well organized, especially at first. We have found this type of drill very popular with pupils when handled by an intelligent teacher who knows what she is trying to do. Sometimes in analyzing a paragraph, ideas will be found that are quite irrelevant to the general topic. Through such analysis the older children at least can be led to recognize this irrelevant material, and it is important that they should be trained to do so. It is Just as important, from the viewpoint of developing study ability, for the child to learn to recognize data not pertaining to the sub- ject under consideration as it is for him to learn to recog- nize pertinent material. Hunting for irrelevant state- ments in paragraphs might profitably be made a special type of silent-reading exercise to be used with reasonable frequency in the supervised-study period. Construction of Outlines Probably there is no more effective kind of supervised study (or silent-reading drill) than is furnished by the construction of lesson outlines in class under the super- vision and with the assistance of the teacher — providing, however, that the children are led to do the major part of the thinking and organizing. Not all material is suitable for this kind of work; but whenever such material is found in the texts, it should be utilized so far as time will permit. The material having been chosen, the first thing to do is to select headings for main topics if good ones are not 266 STANDARDIZED TESTS given in the textbook. Sometimes the headings given in the books are poorly chosen or not well worded. In such cases it is good practice for the class to reconstruct them or find better ones. Then each paragraph should be care- fully analyzed, the relations between ideas studied, and subtopics chosen for groups of related ideas. Finally, the minor details should be grouped under the subtopics which they support. Let us take the before-mentioned article on cocoa to illustrate drill in the construction of an outline by means of paragraph analysis. Here is the article: Cocoa (Ko'-ko) (originally Ko-ko'-a) is a reddish-brown powder obtained by grinding the kernels from the seeds of the cacao, or cocoa, tree. It is widely used in making the popular table beverage known as cocoa. The name, now in general use in English-speaking countries, is a corruption of the more correct form, cacao. The cocoa is a small tropical tree cultivated extensively in Ecuador, Venezuela, Brazil, Saint Thomas Island (off the West Coast of Africa), Ceylon, the West Indies, and Central America. Heat, moisture, and a deep rich soil are the conditions which favor its growth. The straight, regular trunk usually attains a height of twenty feet, and puts forth branches which bear shining, oval leaves, dark green above, and red underneath. The flowers, which have five narrow, bright-red petals, grow directly from the trunk or the older branches, and are almost stemless. The fruit, a cucumber-shaped pod with a thick, deeply-grooved rind, has the same peculiarity. Each pod con- tains many almond-like seeds, covered by a thin, reddish-brown shell, and within each of the seeds is a dark brown kernel, the valuable portion of the plant. The seeds have the commercial name of cocoa beans, while the kernels are called nihs. Most of the work of getting the beans ready for shipment is done by negroes. After the pod is picked, a slit is made in the side with a knife; the pods are then broken open with the hand, SUPERVISED STUDY 267 and the beans and their enveloping pulp are scooped out and carried to a sweating house to go through a process of fermen- tation. This fermentation makes the pulp easily removable, and also improves the quality of the kernel. From the sweat- ing house the beans are taken to sieves or troughs and stirred under water until they are clean and smooth. They are then dried, either in the sun or by artificial means. Finally, in order that the beans may be protected against molds and fungous growths, they are finished, or polished. On some plantations the polishing is done by coolies, who dance upon the seeds until every particle of pulp is removed, and the finished product shines. The beans are then placed in bags or barrels and shipped to the different ports of the world, to be sold to manufacturers. Powdered cocoa, chocolate, and cocoa butter are the chief products of the cocoa beans. In the process of manufacture the seeds are roasted, and the shells removed, and the kernels, or nibs, are placed in a grinding mill with steam-heated rollers. Because of the heat in the rollers the cocoa mass flows out of the mill in the form of a semi-liquid, dark brown paste and can be run into deep pans and allowed to harden. If cocoa is to be made, the mass is remelted and placed in a great press which extracts a large proportion of the fat. The substance is then taken from the press and reduced to a fine powder in a mill consisting of a pair of rollers armed with teeth. Before it is placed on the market the powder is pulverized in a second mill, then is subjected to a thorough sifting. Chocolate is the cocoa mass with the fat left in. If sugar and flavoring are added, the product becomes sweet chocolate. The fat extracted from the cocoa is sold under the name of cocoa butler, and is used as a basis for creams and pomades for the hair and skin and in candy making. The shells of the cocoa beans, usually regarded as a waste product, are sometimes roasted with coffee to add to its flavor, and in some sections peasants use them as a substitute for tea and coffee. 268 STANDARDIZED TESTS Now let us examine each sentence in each paragraph and note briefly what it tells us. Pabagraph I 1. What cocoa is and how it is obtained. 2. Its use as a beverage. 3. Derivation of word "cocoa." Pahageaph II 1. The tree and where it is cultivated. 2. Conditions favoring growth. 3. Shape of trunk and shape and color of leaves. 4 Flowers and where they grow. 5. Description of fruit. 6. Contents of pods. 7. Commercial names of seeds and kernels. Paragraph III 1. Negroes do most of the work. 2. Getting beans from pods. 3. The sweatmg house and its purpose. 4. Removal of pulp. 5. Cleaning the beans. 6. Drying the beans. 7. Polishing the beans. Why? 8. How pohshing is done. 9. Shipment of beans to manufacturers. Paragraph IV 1. Things made from cocoa beans. 2. Things done to the seeds. 3. Product of grinding. 4. Extraction of oil to make cocoa. 5. Grinding of residue to make cocoa. 6. Second pulverizing and sifting. 7. Chocolate. 8. Sweet chocolate. 9. Cocoa butter and its uses. 10. Use of shells. SUPERVISED STUDY 269 Now, with the results of our analysis and the minor details of the paragraphs, we can easily construct our out- line as follows: COCOA I. Cocoa 1. What it is 2. How obtained 3. Use 4. Derivation of name n. The Cocoa Tree 1. Where it is cultivated 2. Conditions favoring growth (a) Heat (b) Moisture (c) Soil 3. Appearance (a) Shape of trunk (b) Leaves A. Shape B. Color 4. Flowers (a) Color (b) Position on tree 5. Fruit (a) Appearance (b) Contents A. Commercial names of III. Preparing Cocoa Beans for Shipment 1. Kind of labor employed 2. Getting beans from pods (a) How accomplished 3. Removal of pulp (a) Where? (b) How? 4. Cleaning the beans 5. Drying 270 STANDARDIZED TESTS 6. Polishing (a) Purpose (b) How accomplished 7- Shipment IV. Manufacture of Cocoa Products I. Chief products 2. Process of manufacture (a) Roasting (b) Shelling (c) Grinding of kernels A. Method B. Product 3- Making cocoa (a) Extraction of oil (b) Grinding of residue (c) Second grinding (d) Sifting 4- Chocolate (a) What it is (b) Sweet chocolate S- Cocoa butter (a) What it is (b) Its uses 6. Use of shells Two or three such outlines worked out in class each term and supplemented by as much seat work along the same lines as may seem practicable will help tremendously in developing the ability to select the main points in a lesson and group the minor details about them. Only by such practice can most pupils learn to recognize and grasp the salient points in what ordinarily appears to them as a dead level of facts to be memorized without discrimination. When the material is so well organized by the author that most of the main thoughts are emphasized by topic sen- tences and other general statements, the outlining is com- SUPERVISED STUDY 271 paratively simple. Such material, however, is the excep- tion rather than the rule. Nevertheless, it should be used, if available, in the first attempts at constructing outlines. When the main topics must be inferred through the association of groups of related ideas, outlining is quite another matter requiring keen thought and analysis. With judicious assistance by the teacher, however, it is not beyond the abilities of fifth-grade pupils. The chief elements of success in this line of work are, (i) intelligent selection of material, (2) careful study of the material by the teacher before presentation to the class, and (3) skill in presenting in order to lead the children to do most of the thinking. Collecting Material as a Phase op Study From constructing outlines of single lessons in a textbook they should be gradually led to the still more difficult and profitable practice of selecting material on a given topic from several books or from other sources and of construct- ing an outline based on this material. In connection with this work they would learn to ,use reference books properly and easily, and also the indexes and tables of contents in other books. Very many pupils even in high school scarcely know that books have indexes, and they are very far from connecting them with any practical use in study- ing. It is a pitiful sight and a sad commentary on the intelligence and diligence of teachers to watch the average pupil try to refer to some particular topic in a book by turning the leaves one at a time and scanning each page for the needed thought instead of turning to the index and locating it in a few seconds. The abUity to organize material gleaned from several sources is particularly important. This kind of work is demanded in colleges and in many high schools. Yet not 272 STANDARDIZED TESTS a few High-school and college students appear absolutely helpless when called upon to do it. They go through books and magazine articles making copious notes, or rather copying verbatim sentences and paragraphs that seem to pertain to their topic. This can scarcely be called note-taking. Then they connect these scraps together with a superabundance of conjunctions without regard to organization, interject a few phrases and clauses to make smooth reading, and submit the result with a sigh of relief to the instructor. Too often they " get by " with this sort of work because so many of them are incapable of doing anything better. Outlines constructed as practice exercises in supervised study should be utilized in composition work. Although many pupils in the Enghsh classes learn to turn out fairly good work in narrative style, most of them do not know what organization means as applied to expository composi- tion. Practice in this sort of work should begin in the upper grades of the elementary school in connection with supervised study and silent-reading work. Teachers are all too prone to declare that such work is beyond the capabilities of elementary pupils, when the real trouble is in neglect or in lack of teaching skill on the part of the teacher. Of course it is beyond them until they learn how to do it. So is most everything else. But how are they going to learn if they are not given a chance to practice under proper conditions. In this chapter I have tried to show that effective super- vised study is not a Utopian dream possible of realization only in the larger and wealthier school systems; but that it is practicable in any school system wherein the adminis- trators and teachers have the courage, the ambition, and the intelligence to attempt it. The fact that supervised SUPERVISED STUDY 273 study, teaching how to study, and silent-reading drill are so closely related that drill in one, when properly con- ducted, serves in some measure for drill in the others, makes invalid any excuses for neglect of any of these highly important phases of school work on the ground of lack of time. INDEX AbiKty in reading, difficulty of meas- ^og, 159- Ability of teachers, how to measure, 14; factors in, 69. Agreement, factors producing lack of teachers, 18. Arithmetic, individual records in, 57; over-emphasis on, 44; sixth grade, paper, distribution of ratings by teachers, 17. Association, keeping the right order of, 163. Average mentality, record of child of, 55. Binet-Simon Intelhgence Scale, 96, 100, 103. Bolenius Readers, The, 201. Bonuses, to teachers, 81. Border-line cases, 107. Bright child, individual record of, 63, 64. Bright pupils, identifying the, iii. Causes of general tendencies, search for, 44. Cave Twins, The, Perkins, 215. Children, teaching how to study, 234. City systems, adaptation of intelligence tests to problems of, 112. Civics, reading material for upper grades, 197. Class, record of, 68. Cleveland Survey Test, $9- Comparing methods, ways of, 84, 86, 91. Conditions revealed by the use of stand- ardized tests, 39; effect upon meth- ods, 94. Content subjects, individual records in, 62. Cooperation of teachers, how to secure, 16. Courtis's Standard Practice Tests, 84, 86, 87, 88. Current events, reading material for upper grades, 198. I Dearborn test, 102. Democracy and Education, Dewey, 148. Diagnostic Language and Grammar Test, Charters, 61. Diagnostic value of tests, 57. Difficulty of liistory questions, as deter- mined by judgment of teachers, 21. Difficulty in teaching silent reading to beginners, the first, 169. Diggers in the Earth, Tappan, 201. Division Scale, Woody, 54, 55. Dull child, individual record of, 65, 66. Educational Tests and Measurements, Monroe, De Voss and Kelly, 12. Efficiency of methods, compared by standardized tests, 82. Efficiency of teachers, measuring, 69, Eliminating the variables, 83. England's Story, Tappan, 196. English, individual records in, 61. English Organization Test, Greene, 43, 46, 61. Equality, fallacy of the theory, 99. Errors of judgment by teachers, 98. Experiment in comparing methods, 86. Factors in efficiency of a teacher, sum- marized, 82. Factors in teaching ability, 69. Fair method of grading pupils, 9, u. Fallacy, of satisfactory progress possible to all pupils, 98. Faulty judgment of pupils by teachers, 97- First steps in teaching reading, 160. First words, teaching the, 162. Geography, reading material for upper grades, 196. Geography Scale. Hahn-Lackey, 55, 61. Good habits of study, how to encourage, 238. Grade I, initiatory steps in teaching 276 INDEX reading, 160; kind o( material needed for, 178. Grading, by mental ages and achieve- ment tests substantially the same, 118; proposed plan of, in rural school, 119. Grading pupils fairly and accurately, 10. Grading purposes, use of intelligence tests for, 30, 114. Graph card, devising a, 32; meaning of, 33; described, 39, 54; tendencies shown by, 42; composite seventh grade, 42; composite fourth grade, 44. Group Intelligence Scale, Otis, 37, 41, 47, 73- Group intelligence tests, lOi. Group, progress, record of, 67, 68. Baggerly Reading Examination, extract from, 183. Haggerty tests, 102, 103. Handwriting, individual records in, 60. History paper, eighth grade, distribu- tion of ratings by teachers, 18. History questions, weights assigned ac- cording to teachers' judgments, 22; ranked by number of times missed on test, 24; summary of ranks and values, 2S- History, reading material for upper grades, 196. Bislory Test, Starch, 55. Borne Life Around Ike World, Mirick and Holmes, 210. Butchinson, Woods, Bealth Series, 196. Illustrated vocabulary, the, 167. Importance of developing efficient readers in the lower grades, 159. Improvement of silent reading, means to, 14s. Inaccuracy of teachers' marks, general, 17- Individual Graph-Record Card, Brooks, 54- Individual Record, samples of, 34, 35, 37, 40, 41. 55, 58. Intelligence Tests, practical uses for, 96. Intensive work, results of, 90. Investigation and readjustments, use of intelligence tests illustrated, 104, lOS, 107. Judgment of pupils by teachers, faulty, 97- Literature, reading material for upper grades, 197; why not adapted for silent-reading drill, 199. Lower grades, reading the most funda- mental subject taught in, 159. Manners and conduct, reading material for upper grades, 197. Marking, finding a satisfactory system of, 13. Marks, general inaccuracy of teachers, 17- Material, collecting, 271. Measuring the Results of Teaching, Mon- roe, 12, 19. Measuring teaching ability by results, 69. Measurement of Intelligence, The, Ter- man, 96. Measurement of pupils, more reliable than teachers' judgment, 98. Mechanics of oral reading, a hindrance to proper reading habits, 148. Median score, 26. Mental ages, and intelligence quotients, 102, 104, 106, 107, III, 114, 116, 118, 121-23. Mental attitude, teacher's, 92. Method vs. Teacher, 82. Methods, compared by standardized tests, 82, 84, 86, 91 ; in teaching how to study, 236; in teaching silent reading. Grade 1, 172, 174, 17s; new vs. old, 93; ways of comparing, 84, 86, 93. Modification of methods and materials, part of a testing program, 143. Moron, the high-grade, 109. Multiplication Scale, Woody, 31. Need of intelligence tests in judging pupils, 97. Objections of teachers to rating by re- sults, 72. Objective measurement, need of, 98. INDEX Observation of teachers by superintend- ent, not a fair way to estimate abil- ity, 14. 277 Oral quizzes, 50. Oral readings, the case against, 150; not necessary, for beginners, 152; the proper time to begin, 154; the value of, 151. Oral-reading drill, 144, 146, 148. Ordinary tests unfair for grading pur- poses, II. Otis Group Intelligence Test, 102, 103. Our European Ancestors, Tappan, 196. Outlines, construction of, 265. Over-emphasis upon arithmetic, 44. Paragraph topic, finding a, 262. Passing mark, 26. Personal judgment vs. objective meas- urement, 98. Physiology and Hygiene, reading mate- rial for upper grades, 196. Pictures, the use of, 167. Plan, for comparing teacher's work under new methods, 93; for teaching silent reading, an effective, 160. Practical notes, upon use of intelligence tests, 96. Primers and first readers, comment upon, 170. Problems of city superintendents, how standardized tests can help, 112. Program of studies, as a guide for pro- motion purposes, 13. Progress, general, individual records in, 62; of pupils, measuring, 12, 52. Punctuation Scale, Starch, 61. Pupils, grading the, 36; how to measure progress of, 12. Pupils' opinions of standardized tests, 134-42- Question Method, the, 253, 254-57. Questions in history, ranked according to difficulty, 20. Eating teachers, plan for, 73, 76, 77, 78; intelhgence tests used in, no. Reaction of teachers and pupils to standardized tests, 124. Reading Aims and Methods, 143. Reading, as a separate subject, in the upper grades, 193; causes of poor re- sults in, 49; new pohcy in, 50; individ- ual records in, 56; material, types for upper grades, 196; scores of fifth grade in, 75- Reasoning Test in Arithmetic, Moru'oe, 59- Results, as the measure of teaching abihty, 69; need of testing methods by, 94; of Mental Tests in a selected school, 120. Retarded children, 109. Riverside Primer, The, 185. Rural school, proposed plan of grading in, 119. Safeguarding tests from pupils, im- portance of, 29. Salary and rating, 80. Schedule of silent reading for upper grades, 195. Science of Everyday Life, The, Van Bus- kirk and Smith, 219. Scores, converted into grades, 31. Second grade, reading in the, 186. Second Readers, comment upon, 176. Sentences, teaching, 165. Silent reading, ability not ensured by drill in oral reading, 144; criticism of current methods of teaching, 157; eflfective plan for teaching, 160; good selections and directions for use, 202- 29; how good methods encourage good study habits, 240, 244; in the lower grades, 159; kind of material needed for, 200; methods actually used, 156; schedule for upper grades, 195; why it should be taught first, 155. Silent-reading ability, essential to study, 239- Silent-reading drill, an opportunity for supervised study, 250. Silent-reading material, ways of using, 230. Silent-Reading Test, Courtis, 131; Mon- roe, 54, 160. Special reading classes for the upper grades, how organized, 194. Special teaching methods, relative values of, 83. 278 INDEX Spelling, individual records in, 60. Spelling Scale, Ayres, 55, 60. Standardized test, used to measure progress, 53; opinion of teachers and pupils, 126-42. Slory of the Greek People, Tappan, 196. Story of the Roman People, Tappan, 196, 222. Study, teaching children to, 15, 234. Supervised study, 244, 246, 262; alter- , nated with recitation, 259; difficulties in connection with, 247; practical method of, 15. Teacher vs. Method, 82. Teachers, how to measure ability of, 14. Teachers' judgments of progress un- satisfactory, 52. Teachers' opinion of standardized tests, 126-34. Teachers' tests as written reviews, 27. Teaching ability, factors in, 69. Teaching how to study, practical method of, IS. Teaching methods, relative values of, 83. Tendencies shown by the graphs, 42. Test, in history, 23. Testing-methods by results, need of, 94. Testing, supervising the, 30. Third Grade, reading in the, 190. Time limit, necessity for, 28. Timing, need of accurate, 27. Trees, Stars and Birds, Moseley, 253. United Slates, The, Winslow, 224. Unjust blame of teachers, 108. Upper grades, reading in the, 193. Usable material in primers and first readers, 171. Values, of teaching methods, relative, 83. Variables, eliminating the, 83. Visual Vocabtdary, Thomdike, 49, 62. Vocabulary building, 51. Vocabulary, the illustrated, 167. War with Germany, influence in devel- opment of group intelligence tests, lOI. Ways of u^ng reading material in the lowest grades, 172, 174, 175; silent reading material, 230. Woody Scales, for measuring ability of pupils, 88, 89. Writing, the situation in, 47.