€-feviaiillatiiil ^^B^fl'^J^'^B® Mwa^wfim Heiu fork HviU QJoUege nf Agticulture At QJntneU UttiWECBitH atljaca, ». 1. Htbrarg Cornell University Library LB 3051.F5 The marking system in theory and practic 3 1924 013 096 916 The original of tliis book is in tine Cornell University Library. There are no known copyright restrictions in the United States on the use of the text. http://www.archive.org/details/cu31 92401 309691 6 THE MARKING SYSTEM IN THEORY AND PRACTICE l&ttraltottal PBgrlfolagg jMottograplfH EbiXti fag QSug MonUaat Whiipplt No. 10 The Marking System in Theory and Practice By I. E. FINKELSTEIN, A. M. (Comeii) (Studies from the Cornell Educational Laboratory, No. 14) WARWICK & YORK, Inc. Salttmorf, 1. &. A. 1913 Copyright, 1913 By WARWICK & YORK, Inc. EDITOE'S PREFACE. When we consider the practically universal use in all educational institutions of a system of marks, whether numbers or letters, to indicate scholastic attainment of the pupils or students in these institu- tions, and when we remember how very great stress is laid by teachers and pupils alike upon these marks as real measures or indicators of attainment, we can but be astonished at the blind faith that has been felt in the reliability of the marking system. School administrators have been using with confidence an absolutely uncalibrated instrument. Only within a very few years have serious attempts been made to scrutinize the theory of marking or to test by statis- tical and experimental procedure the degree of pre- cision that could be expected in its use. What we need to know is: What are the traits, qualities or capacities that we are actually trying to measure in our marking systems? How are these capacities actually distributed in the body of pupils or students? What method ought we to follow in measuring these capacities ? What faults appear in the marking systems that we are now using, and how can these be avoided or minimized? This monograph (originally prepared as a mas- ter's thesis at Cornell University) is a contribution directed toward the answering of these very perti- nent questions. In it the author reviews the conclu- 2 MARKING SYSTEM IN THEORY AND PRACTICE sions reached by previous investigators, sets forth the underlying theories of marking systems, and, finally, demonstrates by a painstaking statistical analysis of the marks given in his own institution what degree of unreliability and what faults of dis- tribution inhere in the ordinary percentile system that is employed in most schools and colleges. These statistical results must not be thought to be peculiar to a particular university, or to universities in gen- eral. They will be found, upon examination, to be the pattern to which the marking system of any edu- cational system will tend to conform, and for this reason this study has not a local, but a general signi- ficance, and the author's conclusions and recommen- dations deserve most careful study by all who are concerned in educational administration. Ithaca, N. Y., April, 1913. G, M. W. CONTENTS. Chaptee I. Page Introductory 5 Chapter II. Theoretical Considerations 9 1. Should marks indicate performance or ability or accomplishment ? 9 2. What is the theoretical distribution of the qual- ities or traits that marks are to indicate?. . . 11 3. What is the best method of translating the dis- tribution into a scale of symbols ? 16 Chapter III. The Distribution op Marks at Cornell University : Combined Results for Numerous Courses 21 1. Marks given in 1902 25 2. Marks given in 1903 25 3. Marks given in 1911 26 4. Combined Curve of Marks in 1902, 1903 and 1911 28 Chapter IV. The Distribution of Marks at Cornell University : Eesults for Individual Courses 37 1. Variation produced by change of instructors. . 39 2. Typical distributions of "high markers" 42 3. Typical distributions of "low markers" 49 4. Peculiarities of distribution in other courses ... 60 5. Marking system of the College of Law 72 Chapter V. Summary and Conclusions 79 CHAPTER I. INTEODUCTOEY. The idea of makiag a careful investigation of the statistical and psychological problems underlying the assignment of grades or marks to students in schools and colleges is of relatively recent date. It is pdthin the last decade tksct serious attention has been paid to such queries as : What should the mark really represent? Should the mark be based upon ability or performance, or even upon zeal and en- thusiasm? "What is the best set of symbols to repre- sent ability or achievement? How are the marks given by different teachers or different schools actually distributed? Is it possible, by exhibition of distributions, or by formal instruction ia the theory of marking, to increase the fairness and reliability of marks? Do students tend to secure the same standing under different teachers in the same school or to maintain their relative standing when proceed- ing from class to class or from school to college f7 From the studies of J. M. Cattell (1905), W. S. Hall (1906), Max Meyer (1908), W. F. Dearborn (1910), A. G. Smith (1911), A. G. Steele (1911), W. T. Fos- ter (1911) and others^ who have discussed various phases of these problems has come the demonstra- tion that few teachers stop to consider what the 'For exact references see the bibliography, p. 85. 5 6 MARKING SYSTEM IN THEORY AND PRACTICE marking system under whicli they work really im- plies ; that the variability in the marks given for the same subject and to the same pupils by different in- structors is so great as frequently to work real injustice to the students ; and that the marking sys- tem in most common use — the percentage system with 100 for a maximum and 60 or 70 as a "pass mark" — is in all probability not the best system. If these conclusions be granted, it is evident that the reliability of the marking system in any institu- -tion of learning is a matter for investigation. If the teachers in the institution are marking unscientifi- cally, or if they are using a system which can be shown to be inferior in theory and in practice, then these facts should be investigated and a remedy sought. Nor may anyone seek refuge in the asser- tion that the marks of the students are of little real importance. The evidence is clear that marks con- stitute a very real and a very strong inducement to work,^ that they are accepted as real and fairly exact measurements of ability or of performance. More- over, they not infrequently are determiners of the student's career. They constitute the primary basis for election to honorary societies, for the award of various academic honors, for advancement from- class to class, for graduation, and may even deter- mine in some measure the student's career after leaving the institution in which they have been as- signed. As Meyer (9, p. 661) remarks, "If different grades were simply means of giving some students notoriety above others, the question would be immaterial, for a gentleman does not seek notoriety. But the 'Colvln (2) shows how the marking system can be used as an in- centive if it is well organized and rational. INTRODUCTION 7 grade has In more than one sense a cash value, and if there is no uni- formity of grading in an institution, this means directly that values are stolen from some and undeservedly presented to others. "The result is that, among the members of the faculty as well as among the students, men look at each other with suspicion. That this attitude is detrimental to the feeling of unity, to the development of a college spirit, is clear to even the most superficial observer. What- ever contributes to a greater uniformity of grading contributes directly towards more peace, a better mutual understanding, a greater commu- nity of purposes among all the members of the institution." The purpose of tlie present study is primarily to examine the distribution of marks as found in vari- ous colleges and classes in Cornell University. But consideration is first given to the relative merits of different marking systems, and to the theoretical considerations which underlie any scientifically or- ganized system. CHAPTEE II. THBOEETIOAL CONSIDEEATIONS. Three theoretical problems deserve consideration before we set forth the data obtained from our in- vestigation of the actual distribution of the marks of Cornell University. The first problem is : Should marks indicate per- formance or ability or accomplishment? The second problem is : What is the theoretical distribution of the quality or traits that the marks are to indicate? The third problem is : What is the best method of translating the distribution into a scale of marks? 1. Should marks indicate performance or ability or accomplishment^ In certain cases, where the examiner has before him merely the results of the examinee's efforts to answer a given set of questions or to solve a given group of problems, as, for instance, in the examina- tions submitted in the Civil Service, it is evident that marks must be based upon performance, i. e., upon work actually done in the examination, without regard to native ability or zeal or previous evidence of acquaintance with the subject-matter of the exam- ination. But in actual school or college work, the teacher has more to guide him than performance in examina- tioB al9ae» He is able, as a rule, to form some idea 10 MARKING SYSTEM IN THEORY AND PRACTICE of the pupil's native ability. He is able, furthermore, to take into account evidence afforded in other ways than by the performance, of the pupil's real knowl- edge and acquaintance with the subject-matter of the course. Thus, he may be convinced that a cer- tain pupil fails to do himself justice m his classroom and examination performance. He may then decide to raise his mark in such a way as to indicate more fairly his accomplishment (as distinguished from mere performance). The issue then takes the form: Shall the pupil be marked according to his ability or according to his accomplishment ? By ability we mean native endowment, intellectual capacity. Accomplishment is certainly very largely determined by ability, but it is also determiued by adequacy of previous preparation, and perhaps still more by zeal and effort. College teachers will read- ily recognize the type of student whose native ability is handicapped by poor training in the preparatory school, and all teachers recognize the type of student whose ability is not reflected in his accomplishment for lack of earnest application. To most teachers it seems axiomatic that marks should indicate accomplishment, and not ability alone. If a capable student shirks his work in physics, he must suffer the penalty of a low mark. If a dull student passes an examination successfully by diut of strenuous application, he is entitled to the credit of his accomplishment. But the opposite position has been defended by some writers. Thus, A. G. Smith (11, p. 384) argues that ' ' College men have greater mental accomplish- ment than the average man, but, measured from the THEORETICAL CONSIDERATIONS 11 standpoint of mental ability or capacity, they are not a group so highly specialized as is commonly believed. This is especially true in America, where the colleges are filled with students drawn from every walk of life." He asserts that college grades, when properly given, should represent ability rather than accomplishment. 2. What is the theoretical distribution of the qualities that marks are to indicate? Let us, for the moment, keep both of these posi- tions in mind. What, now, should be the distribu- tion of marks of a class, first, when the marks indi- cate ability; secondly, when the marks indicate ac- complishment? Native ability, from all the evidence at our com- mand, behaves like any other biological trait. It fol- lows that in any population its distribution is that known as the curve of error, the probability curve, or Gauss's curve. This curve. Fig. A, is a bell- FIGURE A. shaped symmetrical curve, with a mode at the me- dian point, with deviations of equal magnitude and frequency above and below the median, and with a progressive diminution in frequency of occurrence as the deviation increases in magnitude. 12 MARKING SYSTEM IN THEORY AND PRACTICE The question now arises whether this curve repre- sents truly the distribution of ability in high-school and in college students (with whom we are more par- ticularly concerned). The chief factor which might invalidate the application of this curve to a class of any rank above the primary grades is the factor of selective elimination. Evidently, the idiot, the imbe- cile and the moron are eliminated before the high school is reached — the idiot at the outset, the imbe- cile early in the grades, the moron perhaps in the grammar grades or earlier. Undoubtedly, there oc- curs at points in the public school system above the grammar grades a still further elimination of dull pupils from the lower end of the curve of distribu- tion. Those who cannot win promotion in the high school cannot reach the college. If, then, the lower end, at any rate, of the curve of distribution of native ability for the entire population be thus cut off by the machinery of the public schools, it might appear that the distribution of native ability in the college would resemble the curve shown in Fig. B, which is obviously skewed to the left. FIGURE B. Not every one, however, is agreed that elimination of this kind does take place. It would appear that the elimination is not so severe, at any rate, as is THEORETICAL CONSIDERATIONS 13 often supposed. Judd (7, p. 469), for instance, says: "A study of the large high schools of the city of Chicago in its relation to the University of Chicago shows that students go to college from every level of scholarship above the passing mark. ' ' Similarly, Meyer (9, p. 666) writes : "College teachers usually assert that the curve of distribution is not the nor- mal curve, but a skewed curve. . . . They are usually ready to explain this by referring to the elim- ination of poor scholars in the high schools and lower schools. I have considerable doubts as to this elim- ination. Is the work done in a high school really so much like that done in college that there is a large previous elimination of poor college students?" However, the situation, in our opinion, is not quite so simple as this. As the pupil passes from kinder- garten to university the standard of ability presup- posed and exacted for successful work gradually rises. So, also, does the ability of the pupil, and this, merely from maturity, quite independently of his instruction. Along this path of progressive devel- opment of ability arrests occur. The feeble-minded are instances of such an arrest in the lower stages of mental development. Modern psychology teaches us that "retardations occur continuously on up the years of growth to maturity" (Huey, 6, p. 43). The mean ability of a group of children of a given age, therefore, advances with increasing age. The distri- bution of capacity of a thousand children at their tenth year may, therefore, resemble that of the same children at their fifth year, save that the entire group tends to move forward to a higher stage. When the remnant of this same group arrives at college, its arrested members have dropped out by 14 MAEKING SYSTEM IN THEORY AND PEACTICE elimination, but the poorest of the surviving mem- bers represent a minimal ability just sufficient to cross the deadline of university entrance. The dis- tribution, then, of this group of college freshmen might still be that of the normal probability curve, save, of course, that the standard of the whole distri- bution now differs from that of the same group when pupils in the kindergarten, just as the age-norms in the Binet-Simon scale advance progressively in diffi- culty. If these contentions be accepted, the elimina- tion of the academically unfit by the mill of the pub- lic school system does not produce a distribution like that of Fig. B, but leaves us, after all, the form of distribution shown in Fig. A. Hence, if college marks should indicate ability, Meyer, Smith, Dearborn, Cattell and others would undoubtedly be right when they aver that the proba- bility curve should represent the type or pattern of the normal curve for the distribution of university marks, and it would then become a relatively simple matter to lay down rules for the guidance of instruc- tors and for the standardization of the marking sys- tem for any institution, as has been recently done in the University of Missouri. But now let us return to the second possibility, viz., that marks should indicate, not ability, but ac- complishment. Is accomplishment distributed like ability? We have argued that accomplishment is the result of ability, plus previous preparation and zeal. When like zeal or effort is exhibited by two pupils, the one dull and the other brilliant, it seems probable that the gifted pupil reaps a proportion- ately larger result. It is as if effort were multiplied into ability rather than added to it. It is difficult to THEORETICAL CONSIDERATIONS 15 prove this assertion, but if it be admitted, the con- clusion is evident that, if all the students put forth the same amount of effort, the curve of accomplish- ment would be skewed to the right {Fig. C). This FIGURE c. skew would appear in the absence of any marking system. But when a, marking system is arranged to measure accomplishment, another factor undoubt- edly enters to skew the curve further to the right. This factor is the incentive offered to certain groups of students by the critical points of the system. Thus, at Cornell University, where a mark of 60 or above is necessary to pass, there evidently exists a type of student of inferior or average ability who aims for this mark. The possibility of exemption from final examination by attaining a mark of 85 in preliminary examinations offers also a powerful in- centive to students of average ability to push their accomplishment to the extent of their ability. Again, an average student, by persistent effort, may hope to obtain an average of 80 or over, and thus be eligible for consideration for Phi Beta Kappa and other special honors. On the other hand, the brilliant stu- dent, who probably profits most by effort," cannot at- 16 MARKING SYSTEM IN THEORY AND PRACTICE tain a mairk above 100, and it is not difficult for him to obtain a mark of 85 or over. The net result, then, of the imposition of a marking system is to crowd the grades of accomplishment forward to the right, and thus, again, to tend to form a skewed, and not a bell- shaped curve. iFinally, the actual distribution of over 20,000 marks in Cornell University is a curve skewed to the right, as will be shown below (p. 28). If such a curve, compounded of many classes and many ex- aminers, may be thought to represent with great reliability the consensus gentium of the faculty of the University with regard to the distribution of accomplishment, it affords us one more reason for believing that the distribution of accomplishment isi skewed to the right. The pattern or theoretically ideal curve of high-school and college marks is, therefore, not the probability curve, but the skewed curve with the mode to the right of the middle of the abscissa. 3. What is the best method of translating the dis- tribution into a scale of symbols? Given now a pattern or an ideal distribution of ac- complishment, the further question remains: What is the best method of dividing this distribution into groups for translating accomplishment into a symbol or mark 1 Theoretically, there are numerous ways of making such a division, of translating standing into a scale of marks. In actuality serious consideration has been given to a few forms of scale only, viz., the division into two, into three, into four, into five and into one hundred groups. (a) The two-division system. Simplest of all devices is that which divides all students into two THEORETICAL CONSIDEEATIONS 17 groups — "passed" and "not passed." Such a divis- ion is in operation at Cornell University, and at many other institutions in the reporting of the work of students in the Graduate School. While more defi- nite marks may be assigned at the discretion of the instructor, it is customary to report the work of the students simply as satisfactory or imsatisfac- tory. There are some raembers of the faculty who would be glad to see this system extended to under- graduates, but the majority of college teachers, and of students as well, prefer to use a more precise scale, despite the greater labor entailed in the grad- ing of work and ia the keeping of records. It is felt that the student should receive a more exact notion of his accomplishment, and that for many extrane- ous purposes — selection of members of advanced classes, distribution of various awards, etc. — a finer scale is necessary. (6) The three-division system. E. B. Sargent (10, p. 64), who has advocated the placing of a pat- tern curve of distribution on all record sheets to guide the marking of examiners, argues for a three- division system in which the groups are labeled : in- ferior, mediocre and superior. The fundamental merit of this system is the psychologically correct distinction of a large. group of students of aver- age accomplishment, midway between two smaller groups whose work falls short of, or exceeds, this average accomplishment. The defects of this sys- tem lie, first, in the difficulty of distinguisliing fail- ure from mere inferiority which is still entitled to a pass, and, secondly, as A. G. Smith (11, p. 390) has pointed out, "that it furnishes no distinguishing mark of excellence unless the middle group is made 18 MARKING SYSTEM IN THEORY AND PRACTICE SO large as to be open to all the criticism that can be urged against a 'pass' and 'not pass' method of grading. ' ' (c) The four-division system. No one has very seriously defended a four-division system. There is no psychological nor statistical justification for it. (d) The five- division system. The best possible division of the marking scale for any small number of groups is the five-member division. This plan is based upon the orientation of all students around a central group whose accomplishment is construed to be average or medium. The theory of the distribu- tion of ability, and hence in large measure of accom- plishment, teaches us that mediocrity is the com- monest condition. The largest single homogeneous type of student is the average student. Above and below the average lie groups of smaller size contain- ing superior and inferior students — superior and in- ferior with reference to the average group. The five-division system improves upon the three-divis- ion system in that it further differentiates the out- lying groups ; those superior to the average are sub- divided into two groups, the superior and the excel- lent or exceptionally good; those inferior to the average are subdivided into inferiors and failures.^ The theory of the application of the five-division system to the actual grading of students assumes that the actual distribution of marks should conform 'For purposes of administration it may be thought desirable further to differentiate between 'conditioned' and 'absolutely failed' — following the custom of many institutions of permitting the former to try for a 'pass' by taking a 'make-up' examination, but compelling the latter to take the course again in its entirety in order to secure credit. In prac- tice the scale would then contain six symbols, but it would, neverthe- less, be in theory a five-division system. THEORETICAL CONSIDERATIONS 19 fairly closely to a theoretically predetermined dis- tribution. The question as to what this predeter- mined distribution should be in the case of a skewed curve will be treated later (pp. 30-33). We may set forth here, however, the plans proposed by certain psychologists for the translation into marks of the bell-shaped curve of distribution. If we assign the symbols A, B, C, D and E to excellent, superior, average, inferior and failure, respectively, the divi- sions recommended by Professors Meyer, Dearborn and Cattell for each hundred students are as follows : A. B. o. D. E. Meyer 3 22 50 22 3 Dearborn 2 23 50 23 2 Cattell 10 20 40 20 10 In the opinion of these writers, then, from 40 to 50 per cent, should be marked average, from 20 to 23 per cent, should be superior, and inferior to the average, and from 2 to 10 per cent, should receive the highest mark and the like number should fail. (e) The percentile system. It is not very diffi- cult to grade students on the five-division system. Is it possible to say as much of the system which many institutions follow, according to which marks are based on a scale of 100 points? Theoretically, this scale implies that distinctions of a fineness of one-hundredth may be made, and in practice such distinctions are constantly attempted. But what is the difference, if any, between a mark of 75 and one of 76? What, for that matter, does 75 mean? Has the student accomplished 75 per cent, of some ideal accomplishment? It is a commonplace of statistics that a scale whose units are not defined or whose units are not identical throughout is no scale at all. 20 MARKING SYSTEM IN THEORY AND PRACTICE The fact that different instructors place a different interpretation upon the symbols of the percentile system is evidence enough that it is not the scientific measuring rod that it pretends to be. The very fact that its divisions are so minute is doubly insidious ; it promises precision, but it cannot afford it.^ In short, the 100-division scale has no psychological justification. On the other hand, the five-division system, which is evidently based on a different plan altogether, is simple to use, and the results of each instructor are easily checked at any time.^ 'Since this was written, Professor Starch of the University of Wis- consin has reported the results of a study upon the "Reliability and Distribution of Grades" (Psychol. Bulletin, 10, Feb. 15, 1913, p. 74), which shows that the marks assigned by more than one hundred teach- ers to two papers in English and one in geometry have a probable error of from 4.0 to 7.5 points on a percentile scale. Starch concludes : "The steps on a scale should be at least twice the size of the mean variation or probable error of the measurements in order to be dis- tinguishable steps. Hence the steps on a marking scale should be at least two times 4.2, or approximately 8 points. And hence on a scale of passing grades of 70 to 100 only four steps can be used with any degree of objective reliability." This statistical conclusion, then, con- firms very prettily our argument for a five-division scale — four marks above the passing limit and one mark below it for failures. ^See, for example, the very interesting history of the establishment and operation of a five-division system at the University of Missouri, as narrated by Meyer (8, 9), CHAPTEE III. THE DISTKIBXJTION OP MAEKS AT COENELL UN'IVEESITY : COMBINED EESULTS FOE NUMEEOUS COUBSES. The purpose of this chapter is to show the actual distribution of marks at Cornell University when a large number of different courses are combined. The effect of combining the marks of several thou- sand students in numerous courses is, naturally, to eliminate or to cancel chance variable errors in the marking. The resultant curve thus compounded of the marks given in varied classes is as true a picture as can be obtained of the actual distribution of ac- complishment of students as judged by the instruct- ing staff. Whether the distribution may not be affected by certain constant errors is a matter to be discussed a little later. The material at our disposal for securing these combined results comprises three sets of data. The first set represents the marks given in the College of Arts and Sciences during the first term of 1902- 03 to 5396 students in 66 courses. The subjects represented are Latin, German, French, English, Philosophy, Psychology, Education, History and Political Science, Mathematics, Physics, Chemistry, Botany, Invertebrate Zoology, Physiology and Geol- ogy. The second set of data represents the marks given in the same College in the following academic 21 22 MARKING SYSTEM IN THEORY AND PEACTICH year. It includes the same courses and 7522 stu- dents.^ The third set of data represents the marks collected by the writer and shown in detail in Chap- ter IV. It includes 7430 marks in 31 courses in dif- ferent colleges of the University (Arts and Sciences, Agriculture, Mechanical Engineering and Civil En- gineering), and also 711 marks in three courses in the College of Law. These three sets of data will be referred to for convenience as the 1902 marks, the 1903 marks and the 1911 marks.^ We shall present the distribution for these three sets of data in the order given, then combine them into a single distri- bution and discuss the form of this final compounded curve. TABLE I. Collective Distributions. Showing the Per Cent, of Students in the Several Groups. No. of Year. Marks. 0-39. 40-44. 45-49. 50-54. 55-59. 60-64. 1902 5,396 1.6 1.7 1.6 3.4 2.3 11.8 1903 7,522 1.3 1.2 2.1 3.5 1.8 10.2 1911 7,430 0.6 0.4 1.1 2.7 2.2 12.8 All three 20,348 1.2 1.1 1.6 3.2 2.1 11.6 All three. (Revised).... 20,348 1.2 1.1 1.6 8.2 5.6 8.1 Ex- Tear. 65-69. 70-74. 75-79. 80-84. 85-89. 90-95. 95-100. empt. 1902 10.2 12.9 15.8 15.6 11.5 8.9 2.7 23.1 ^ 1903 10.0 13.6 16.0 16.2 11.6 8.7 3.8 24.1 > 1911 12.5 15.4 16.4 12.7 14.0 7.2 2.0 23.2 i All three 10.9 13.9 16.1 14.8 12.4 8.3 2.8 23.5 All three. (Revised) 10.9 13.9 16.1 14.8 12.4 8.3 2.8 23.5 'My thanks are due to Prof. W. F. Willcox of Cornell University for the use of these two sets of data which were compiled under his direc- tion several years ago. ^As explained in Chapter IV, this third set of data contains marks extending backward from June, 1911, for a length of time necessary to secure at least 200 marks for each course. In most cases the period represented falls between 1910 and 1911. RESULTS FOR NUMEROUS COURSES 23 To make the charts intelligible a word of explana- tion is needed. The percentile system is used at Cornell — save in the College of Law, of which we shall speak more definitely below. The 'pass mark' is placed at 60. A mark between 40 and 59, inclusive, is known as a 'condition' — the student is entitled within one year to try a 'make-up' examination in order to gain credit for the course, provided he reaches 60 or above in this examination.^ A mark below 40 represents complete failure; the student must take the course again in its entirety, and must then pass at 60 or over in order to gain credit for his work. In certain courses students may, at the option of the instructor, be exempted from the final written examination. The mark which must be at- tained to secure exemption varies in different col- leges, save that in the College of Arts and Sciences exemption, if given at all, must be based upon a mark of 85 or over. In the College of Mechanical Engineering there are no special or formal final ex- aminations ; the mark is based upon the work of the students during the term. All the charts of distribution are plotted with the scale of marks as the abscissas, and in units of 5 (or 6) points each, with the highest marks at the right. Thus, the division at the extreme right represents the six marks 95, 96, 97, 98, 99 and 100. The next division represents the five marks from 90 to 94, in- clusive, and all subsequent divisions on toward the left end of the abscissa each represent five marks, save that all marks including 39 and below are in 'Through oversight, it wos not noticed until too late to make the correction that the mark of 40 is counted as complete failure, not as conditioned. This error does not affect the conclusions in any way — Editor. 24 MARKING SYSTEM IN THEORY AND PRACTICE the single division at the extreme left. The brdi- nates in all charts represent the per cent, of students receiving the marks in the various divisions of the scale. In all the charts, furthermore, two additional fea- tures of the distribution are shown. First,- the per- centage of students who Would be exempt from the final examination, on a basis of 85 or over (assum- ing exemption were permitted in all classes), is shown in the accompanying legends. Secondly, the range of marks received by the middle 50 per cent, of the students is shown on each graph by the size and position of the solid black area. This range is found by counting off 25 per cent, of the marks from the upper, and 25 per cent, of the marks from the lower end of the total distribution found in each case.^ Since, by theory, the middle 50 per cent, of any group of students must be neither brilliant nor dull, but simply straightforward, average students, the distribution of this group is of special interest. Speaking generally, we should not expect the marks obtained by this group to spread over a large range, since they represent a homogeneous group in point 'To avoid possible misunderstanding, a word should be said concern- ing the boundaries of the black area on the charts. The upper and lower limits of the middle half of the students need not necessarily, of course, coincide with a division point upon the abscissa. Thus, in Chart I, the upper limit of the middle group lies in the range 80-84, but there are also some students in the upper 2.5 per cent, whose marks fall within the same range. To be specific, since 23.1 per cent, lie in the range 85-100, 1.9 per cent, of the students superior to the average lie in the range 80-84 per cent. Accordingly, the vertical column erected as an ordinate over the range 80-84 is blackened to within 1.9 units of its tip only, and not fully to the tip. Similarly, a mark lying be- tween 65 and 69 is for the most part obtained by students belonging to the middle or average group, but there are a few students inferior to the average (on our theoretical definition of average) that also obtain marks within this range. RESULTS FOR NUMEROUS COURSES 25 of ability and accomplishment. Again, we should not expect an average student to attain exemption from final examination, nor, on the other hand, to run any grave risk of being 'conditioned.' ZO— ~ College of A-rts and Sciences in Ooz' 66 Courses 5396 Students ^^_ I0.6 Below Pass ,„, ZS.iXExempi ,- l-LX 20- 15 — ro— 5— O 40 45 50 55 60 65 70 75 80 65 30 95 ZOO College of Arts and Sciences in isos 66 Courses 75ZZ Students 9. 9 / Below Pass ;54. 1 Z Exempt 0- CU 40 45 50 55 60 65 70 75 80 85 90 95 lOO CHARTS I AND 2. Combined Disteibution of Masks in 66 CouESES IN 1902 AND 1903. "The L/nivei?3ity in I9i(. 3' COUKSE5 7-430 SruoeNTS. 7^Below"B\33 ZS.Z^ E; £ ScrENces TvpicAL High Marker 1.5 Z Below Rvas 70-0 J Exempt O 40 45 50 55 60 65 70 75 60 65 90 95 100 CHART 10. DiSTEIBUTION IN COUBSE C. 44 RESULTS FOR INDIVIDUAL COURSES 45 the original marks above 90, a larger increment to those between 80 and 89, a still larger one to the marks between 70 and 79, and so on. No marks are given between 55 and 59. A very small number of students are conditioned, and a still smaller number, practically negligible, are failed, A definite process of advancing medium-grade students to the exempt limit is also displayed. The result is clearly to pro- duce an array in which the marks of 60 to 84 occur with almost the same frequency, in which the marks of 85 to 89 are given with disproportionate fre- quency, and in which altogether too many students are credited with an accomplishment of 90 or over. Exemption is granted to 70.8 per cent, of the class. Average students range between 80 and 94 in their marks. Chart 11 shows an array derived from a course in the College of Mechanical Engineering. The pri- mary explanation of the extremely favorable marks here is found in the reputation of the course as a 'snap,' to use students' parlance. It is also possible that the conditions under which the work is done may tend less than ordinarily to check cheating on the part of dishonest students. Whatever factor is at work, the result is that no student is conditioned or failed, that all average students would be entitled to exemption on the 85 basis, and that 93.6 per cent, of the students obtain a mark of 80 or over! The remedy apparently is to condense the work into shorter compass or to incorporate the material, since it is so simple, in various other courses. The nature of the subject-matter suggests, at least, that it might well be included in the work given to the same students by another department. 40. 35- CoLLEGE 6F Mechanical Enoineer'ing TVpcAL HiQH Marker ^5_ None Below Fa35 76.;?% Exempt zo- 15—1 5- O — O 40 45 so 55 So 65 70 75 60 65 90 SJ POO CHART 11. DiSTBIBUTION IN COTJBSE D. 46 RESULTS FOR INDIVIDUAL COURSES 47 The most extreme case of high marking which came within the scope of this investigation is Course E, shown in Chart 12. The course is one in the Col- lege of Arts and Sciences. The work consists of lec- tures, outside readings and textbook study. The outside reading is not specifically prescribed, but must simply cover a given amount of time. In many cases the students read books that chance to interest them, but which do not bear directly upon the sub- ject-matter of the course (though dealing, to be sure, with the general field of which the course forms a part). The lectures practically give everything ifound in the textbook, so that the latter might really be dispensed with by the student without affecting his standing. In addition, the professor in charge undoubtedly overestimates the accomplishments of the students, or, what amounts to the same thing in the end, sets a low standard of performance. The result is that no one is ever failed or conditioned, that 65 is the lowest mark given, that 78 per cent, iwould be exempt from the final examination, and that students of average ability are sure of a mark between 85 and 94.^ When we state that these four courses (B to E) are not selected, small-sized advanced classes, but large groups of undergraduates in elementary work, we can find no sufficient justification for the charac- teristics so strikingly exhibited in their graphs. Imagine groups of undergraduates, assembled at random from the student-body as they are in these 'The tendency to mark high is inherent in human nature. Dr. RufE- ner says : "A temporizing professor who loves popularity, ami desires, like the old man in the fable, to please everybody, is sure to be guilty of this fault, and, like many a politician, to sacrifice permanent good for temporary favor." -»5 — 40 -^ 55- 30- Z5- 20- 15- 10- 5- CoLLEQE OF Arts ^SCIENCES)' The HroHEST Marker None ■BeiowBiss 78^ Exempt 40 45 50 55 60 65 70 75 80 85 SO 95 lOO CHART 12. DiSTBIBTJTION IN CoTJKSE E. 48 RESULTS FOR INDIVIDUAL COURSES 49 courses, who, year after year, display such extraordi- nary accomplishment ! The marks charted in these four courses represent, collectively, 909 students, of which one unfortunate failed and two were con- ditioned. If all the courses of the university were patterned after these four, and if 85 were recognized, as it is now, as a mark of merit, then three out of four students would be entitled to election to the sev- eral honorary societies that seek for students of merit for enrollment in their organization. 3. Typical distributions of 'low markers.' A selected group of 'low markers' is displayed in Table IH, Courses F to M (Charts 13 to 19), with the exception of Chart 14, which is placed here for the sake of comparison. Chart 13, which is perhaps the most extreme in- stance of low marking in our material, is a certain course iu the College of Arts and Sciences which is TABLE III. Effect of Zeal and Disteibutions of Low Mabkebs. Showing the Per Gent, of Students in the Several Groups. No. of Course. Marks. 0-39. 40-44. 45-49. 50-54. 55-59. 60-64. 65-69. 70-74. F 266 1.1 ... .7 5.3 ... 27.0 32.0 12.4 G 353 1.7 7.3 .3 11.9 12.3 17.8 H 226 3.1 1.8 .9 4.4 .4 26.1 16.3 13.3 1 235 2.1 ... .9 1.7 4.2 23.4 14.5 19.2 J 234 2.1 .9 2.1 3.8 .4 21.8 16.7 19.1 K 317 1.2 2.5 2.5 5.7 5.7 18.3 9.5 12.0 L 208 .5 .5 1.9 3.8 1.9 18.7 10.2 16.4 M 273 1.5 2.9 6.9 17.6 17.9 22.7 Ex- Course. 75-79. 80-84. 85-89. 90-94. 95-100. empt. F 16.5 3.5 1.5 1.5 G 19.8 15.0 11.9 2.0 ... 13.9 H 12.9 9.3 7.1 2.6 1.8 11.5 I 13.6 7.2 12.3 .9 ... 13.2 J 8.2 6.1 14.1 4.7 ... 18.8 K 9.8 6.6 16.7 6.6 2.9 26.2 L 14.9 9.6 12.1 8.1 1.4 21.6 M 13.6 12.9 2.9 1.1 ... 4.0 35 — 30- zs- zo— 15- CoLLEGE OF Arts and Sciences Course Given to Engineering Students 7.1 X Below Pass \.S% Exempt 5- — O 40 -45 50 55 GO 65 70 75 8o 85 90 95 CHART 13. DiSTKiBUTioN IN Course F. 60 ao- 15- 10- 5- 0- CoLLEQE OF Arts and Sciences Course Given to Akts Students 9.3 2[ Be-low B^ss 13.3 ;?[ Exempt O 40 45 50 S5 60 65 70 75 SO 85 90 95 lOO CHART 14. DiSTBiBUTiON in Course G. (For comparison with Chart 13, Course F, on the same subject.) 51 52 MARKING SYSTEM IN THEORY AND PRACTICE prescribed for students in engrneeriag. For com- parison with Chart 13 we introduce here Chart 14, which does not happen to belong to the low-marker group, but which is a course on the same general topic, in the same department, though given by an- other member of the instructing staff of the depart- ment and primarily to arts students. This latter course is more theoretical; the former, that given to engineers, is more practical and adapted to the immediate problems of the man in business. It is generally conceded that the former course is the easier, but the distribution of marks shows much poorer accomplishment in it. This situation is in- teresting enough to demand a moment's attention. The question arises: Why is the distribution of Chart 13 so different from that of Chart 14? The instructors who assigned these marks have been ac- customed to work together in other courses, and their standards, so far as can be judged, are not dis- similar. Again, it is impossible to argue that the engineering students are, as a group, inferior in ability to the arts students. There is left, then, the factor of zeal or training. Evidence furnished by the testimony of numerous students and corroborated by the opinions of the instructors themselves, makes it clear that the engineering students, as a group, look upon this prescribed course as a 'necessary evil.' They take but a half-hearted interest in it, and, for the most part, strive merely to 'get through. ' The result, as Chart 13 shows, is that no student receives a mark above 89, that only 1.5 per cent, are given marks of 85 or over, that the most frequent mark lies between 65 and 69, and that the middle half of the class fall between 60 and 74; in 45— Low-Marker vs HroH-MARKER Showing Vai?ia6ii.ity of Distribution 35- zs- 20- 15- 5- o- I -K) 45 50 55 60 65 70 75 SO 85 90 95 lOO CHART 15. A Combination on the Same Base Line of Charts 12 and 13. 53 54 MARKING SYSTEM IN THEORY AND PRACTICE other words, fall, as a group, entirely below the aver- age of the University. Another point of interest: in both Chart 13 and Chart 14 there is displayed a decided aversion to giving a mark between 55 and 59 ; it is given but once in 619 cases. The examiner states that, in the sub- ject in question, it is difficult to grade a student much closer than 5 points ; and that it is impolitic to invite an argument over a mark between 55 and 59. This is further evidence of the impossibility of living up to the implications of the percentile system. We pause here, before passing to other instances of low marking, to call the reader's attention to Chart 15, which, because it combines upon the same base-line the distribution of Chart 12 (high marker) and Chart 13 (low marker) will serve to picture graphically the inequalities of the regulation mark- ing system in actual practice. This chart preaches its own sermon, so that further comment is unneces- sary. The marks displayed in Chart 16 are derived from a course in a department which has the general reputation of being the hardest marking department in the College of Arts and Sciences. Attention may be called to the mode at 60-64, to the progressive diminution of marks from 60 upwards, and to the fact that 10.6 per cent, of the students are condi- tioned or failed. A student of average ability can- not secure a mark above 79, while he may barely es- cape being conditioned. Charts 17 to 19 (and likewise Course L, not shown graphically) are all examples of relatively low marking, combined with an interesting tendency to distribute the marks so that three modes appear. 30- 25- ZO- (5- 10- 5- D- CoLLEQE OF Arts i Sciences Typical Low^Markek lo. 6 2 Below Pass ii.s^f Exempt 40 45 50 55 60 65 70 75 80 85 30 35 100 CHART 16. DiSTBiBUTiON IN Course H. 55 Z5— 20- 15 — 10 — 5 — 0- COLLEQE OF AltTS :£ SCIENCES 8.9 2 Below Pass I3.Z% Exempt O 40 45 50 S5 60 65 70 75 60 35 90 95 100 CHART 17. DiSTKiBUTioN in Couese I. 56 Z5- 20- 15- 10- - College of Arts and Sciences g.'S ^ Below Pass 18.8% Exempt ■«) 45 50 55 60 65 70 75 80 85 90 95 100 CHART 18. Distribution in Couese J. (Note the similarity to Chart 17, another course by the same instructor.) 57 58 MARKING SYSTEM IN THEORY AND PRACTICE In all of them the primary mode is at the 60-64 range; with one exception, the secondary mode is at the 70-74 range, and the tertiary mode at the 85- 89 range. In other words, the commonest mark as- signed is that which just permits the student to pass. The next most common marks are those which as- sign the students to a rank close to the average per- formance of the whole student body, while the next most frequent mark is that which entitles the student to exemption. Since more students barely pass than reach the average mark, the curves are all skewed to the left. Charts 17 and 18 are of special interest, because they represent two different courses in the College of Arts and Sciences that are given by the same pro- fessor. Their similarity is striking, and is a good example of the influence of the personal equation in the distribution of marks. In Chart 19 the distribution of Course K repre- sents what may be termed a ' staff mark. ' The course in question, given in the College of Arts and Sciences as a prescribed course to students in engineering, is divided into a large number of sections, and taught by a corps of teachers. The subject-matter of the course and the final examination are the same for all sections, but the marking of each section is in charge of its own teacher. All these marks are assembled in the one distribution. The curves show modes at the 60-64, 85-89 and 70-74 ranges; the average stu- dents' range is evidently too large (60 to 89) ; des- pite the mode at the 60-64 range, 26.2 per cent, of the class attain a mark of 85 or over. On the other hand, 17.6 per cent, of the students are conditioned 20- 15- 10- 5 — COLLEGE OFATfTS I SCIENCES 17.6 Z BELOW Pass ze.Z ^ Exempt 40 45 50 55 60 65 70 75 80 85 90 95 ;00 CHART 19. DiSTEIBUTION IN COUBSE K. 59 60 MARKING SYSTEM IN THEORY AND PRACTICE or failed. In short, the curve of distribution is too 'flat.' Course L^ shows practically the same shape of distribution as Course K. It has the three modes at the same ranges as the last chart, and has 21.6 per cent, of the class receiving exempt marks. The aver- age student falls in the range from 60 to 84, and 8.6 per cent, of the students receive either 'condition' or 'failure.' The distribution of Course M' is less decidedly that of a severely marked group. The mode falls at 70-74 rather than at 60-64, and the distribution falls away progressively on either side of this mode, as it should. However, the percentage entitled to ex- emption (4 per cent.) is quite small, and no member of the class gets 95 or over. This is a course in which the lectures are given by a series of different speak- ers, and the examination papers are marked under the supervision of a single man, who has general charge of the work of the course. 4. Peculiarities of distribution in other courses. The remaining distributions, in which the mark- ing is neither decidedly low nor decidedly high, have been divided into two groups, the first comprising unimodal, the second multimodal distributions. (a) The unimodal distributions. Courses N to T (see Table IV), have the merit of conforming, at least to some extent, to the theoretical distribution, according to which mediocrity or average accom- plishment is the most frequent type. In the first four of these courses (only one of which. Course P, is here shown graphically), the range of marks as- 'Not here reproduced graphically. See Table III for details. 25— ZO— 15— 10 — 5 — College, OF Arts ANDX Below Pass iZ.Z^ Exempt O 40 45 50 55 60 65 70 75 ao 85 30 95 100 CHART 20. DiSTEiBUTiON in Couese P. 61 62 MARKING SYSTEM IN THEORY AND PRACTICE TABLE IV. Unimodal Disteibutions. Showing the Per Gent, of Students in the Several Qroups. No. of Course. Marks. 0-39. 40-44. 45-49. 50-54. 55-59. 60-64. 65-69. 70-74. N 211 .9 .5 ... 2.4 3.3 8.1 12.8 18.5 305 .7 .3 .7 1.6 1.9 13.1 14.8 16.4 P 221 .4 ... .4 1.8 3.7 14.5 14.5 20.8 Q 328 ... .3 1.2 3.9 4.6 9.5 17.1 18.0 R 254 .8 .8 1.9 .8 6.0 8.7 17.3 23.2 S 254 .4 1.5 .8 6.3 9.5 20.1 T 112 9 2.7 9.8 Ex- Course. 75-79. 80-84. 85-89. 90-94. 95-100. empt. N 27.5 17.5 7.1 1.4 ... 8.5 18.4 18.0 9.5 3.6 1.0 14.1 P 18.5 13.2 5.9 5.9 .4 12.2 Q 20.1 12.2 11.6 1.5 ... 13.1 R 28.8 7.9 3.5 3.5 S 27.2 13.4 11.0 8.3 1.5 20.8 T 11.6 38.4 29.5 4.4 2.7 36.6 signed to students of average accomplishment is the same, viz., 65 to 84. In Course E the range is con- tracted to 65 to 79 ; in Course S advanced to 70-84, and in Course T to 80-89. In Courses E and S there is a slight irregularity at the 55-59 range, but the disturbance is too slight to disbar the distributions from the unimodal group. It is perhaps no accident that, with one exception (Course T), these unimodal distributions are derived from courses in pure or applied science. , (&) In our second group, the multimodal curves, are included Courses U to DD. (See Table V). In details they vary considerably. We shall pass over 25 — ZO- 15 — 5 — O— ( College of Arts and Sciences 7.Z % Below Pass 9.9 % Exempt O -K) -45 50 55 60 65 70 75 80 e5 90 95 CHART 21. DisTBiBTJTioN IN Course U. 03 zo— 15— 10 — 5- 0- CouLEGe OF ,4rts and Sciences 10.5 Z Below Pass. ie.3/ExEMPr 40 45 50 55 60 65 70 15 60 £5 30 95 100 CHART 22. DisTBiBUTioN in Course V. 64 RESULTS FOR INDIVIDUAL COURSES 65 them rapidly, calling attention to features of interest in some of them. TABLE V. Multimodal Distributions. Showing the Per Cent, of Students in the Several Groups. Course. U. V. w. X. Y. No. of Marks. 0-39. 40-44. 45-49. 50-54. 55-59. .4 1.1 3.0 2.3 Z... AA. BB. CO. DD. 262 191 207 225 251 195 232 215 228 198 1.1 3.7 1.4 .8 .5 .5 2.9 1.8 5.2 .4 1.5 Course. U. V. w. X. Y. Z... AA. BB. CC. DD. .5 1.4 1.8 1.0 75-79. 18.4 19.8 17.4 15.1 12.7 30.3 12.9 18.7 17.1 15.1 1.3 5.5 .4 4.1 80-84. 11.8 10.4 4.3 7.1 9.6 22.1 28.1 11.2 16.3 13.2 3.1 .5 4.9 3.2 2.8 2.2 .5 85-89. 9.1 13.1 19.3 18.6 14.3 30.8 14.6 7.0 19.2 17.2 60-64. 16.5 16.7 15.9 17.8 14.7 1.5 8.3 15.7 3.1 10.6 90-94. .8 4.2 2.9 11.6 6.4 2.0 3.9 1.0 11.0 7.5 65-69. 13.4 9.5 16.5 11.1 13.7 1.5 10.3 19.6 7.9 13.2 95-100. 1.5 1.8 .4 2.2 4.5 70-74. 22.8 14.3 18.4 10.2 18.6 9.8 20.6 16.3 18.0 11.6 Ex- empt. 9.9 18.8 22.2 32.0 21.1 33.3 18.5 8.8 32.4 29.2 Chart 21 is the array for a course in the College of Arts and Sciences, whose professor is generally reputed to be a fair marker. His curve of distribu- tion varies, however, from the pattern we have rec- ommended ; first, in that it shows a general tendency to fall short ia the frequencies in the upper end of the scale (no mark above 94, only 9.9 per cent, exempt, and no average student gaining a mark above 79) ; second, in that it shows a disproportion- CoLCtGE OF Arts and Sciences S.ZZ Below Rvss ZZ.zX Exempt 20— 15 — 10 — 5— • o- 1=1 O -K) 45 50 55 eo 65 70 75 80 85 90 95 /OO CHART 23. DisTEiBUTiON in Couese W. 66 RESULTS FOE INDIVIDUAL COURSES 67 ate tendency to give marks between 60 and 64. Some of the marks in this range should have fallen in the 55-59 range, and some probably in the 65-69 range. Chart 22 resembles Chart 19 in being a composite of marks for several sections of the same course, the present distribution being that of a language course in the College of Arts and Sciences, which is run in four sections. The two distributions have a certain amount of similarity, notably in the low frequencies assigned to the middle range of accomplishment. Medium-grade students range in marks from 60 to 84. The present curve has three modes, at 60-64, 75-79 and 85-89, respectively. The peculiarity of Chart 23 (Course W), a science course, is in the curious 'hole' at the 80-84 section of the scale. To counterbalance this failing there ap- pears a second curiosity — the range 85-89 forms the primary mode of the distribution. Again, despite the fact that 22.2 per cent, reach the grade of 85 or over, no one exceeds 94. Finally, the frequency of the marks between 60 and 79 is virtually constant for each section of five points. Another language course is shown graphically in Chart 24 (Course X), which displays considerable irregularity and deviation from the distribution to be expected. Exemption is rather freely accorded (32 per cent.), and average students are spread over a range from 65 to 89. This failure to perceive the homogeneous character of this group of medium worth is the fundamental defect of the curve. A fondness for 60-64 is another evident characteristic, due, evidently, to pushing over the 60 mark those who should be conditioned, only 15 out of 225 are conditioned, while no one fails. 20— 15- 10- 0- CoLLEGE OF /Arts and Sc/ence5 6.7 % DelowPass 3Z% Exem'K? 40 45 50 55 GO 65 70 75 ©0 65 90 95 (00 CHART 24. Distribution in Cotjese X. 68 30- 25- 20- 15- lO~ 5 — — ConEGE oc Mechanical Engineering I.? 5; Below Pa53. I8.5jf E'xempt ' JUL 40 45 50 55 60 65 70 75 30 85 90 95 lOQ CHART 25. DiSTBiBTJTiON in Cotjese AA. 70 MARKING SYSTEM IN THEORY AND PRACTICE A somewhat similar criticism could be passed upon Course Y (not reproduced graphically), a course in the College of Arts and Sciences, in which too many 80-to-84-grade students are pushed for- ward to exemption, and in which the 60-64 range is also too frequently used. A member of the faculty who nearly deserves to be grouped in our second division, the liigh-markers, is responsible for Course Z (not reproduced, save in Table V). Exemption is gained by one-third of the class ; only 1.5 per cent, of the students are failed or conditioned, while the average student never falls below 75, but may, indeed, win a mark of 89. The course in the College of Mechanical Engineer- ing (Course AA), displayed in Chart 25, is not one that should cause worry on the part of the student. While no one gets above 94, yet no one fails, and but 1.3 per cent, of the students are conditioned. The average student ranks between 70 and 84. Here the tendency is apparently to avoid the range 75-79 in favor of the range 80-84. Course BB (see Table V) is from one of the large introductory courses in pure science. In general, the distribution inclines towards the lower marks, so that only 8.8 per cent, would reach exemption ; no one exceeds 94, and 65-69 is the mode. Another large elementary course in science is dis- played ui Chart 26 (Course CC). Save for a too high number of those exempted (32.4 per cent.), the distribution is one of the best found in our data. In Chart 27, from the College of Arts and Sciences, the examiner has avoided the 55-59 range. The proportion exempted (29.2 per cent.) is too high, but the general form of the distribution is ro- CoLLDSe OF ARTS AND SCIENCES 5.2 J' Below T5»ss 3Z.4;jf Exempt O -W -« 50 55 so 65 70 IS 80 85 90 95 loo CHART 26. DisTBiBUTioN in Coubse CC; 71 72 MARKING SYSTEM IN THEORY AND PRACTICE otherwise not bad. This chart may be profitably compared with Chart 12 (p, 48), as the two come from the same department, apply to the same stu- dents, but are given by two different professors. The difference in the standards of accomplishment held by these two members of the faculty yields further evidence, if such be needed, of the inequalities that prevail in the marking system at present in force at Cornell. 5. The marking system of the College of Law. In the College of Law there prevails a system of marking that is radically different from the percen- tile system that we have just been discussing at length. The Law School system embraces six dis- tinct marks, but, unfortunately, these six marks do not conform, either in intention or in practice, with the restricted-unit systems that were discussed in Chapter II. Nor does it appear that any effort has been made by those who introduced this system to relate it definitely to the percentile system which it replaced, and which still prevails elsewhere in the University. The translation into the regular Uni- versity percentile system of the six symbols — EE, E, G, P, P-60 and Cond. — ^which are in use in the Col- lege of Law, has been arranged in the charts which follow, in accordance with the statements of their values, as furnished by the Dean of that College.^ Law School Marks Cond. P-60 P O E EE Approximate Equivalents. . .Below 60 58-63 60-74 75-89 90-98 99-100 ^It appears that a mark (in the numerical system) of 60 might be either P-60 or P in the College of Law. It is explained that the papers are marked in numerical terms and then translated into the six symbols. If a final paper warranted 62, the instructor would report the paper as P if the class work was acceptable, but as P-60 if both class work and final examination were Inferior. X,o— 15 — College of Arts and ScrENCES 7. 1 Jf Below Bxss Zs.zZ Exempt 5 - O 40 .45 50 55 60 65 70 . 75 SO 83 90 95 (00 CHART 27. DiSTEiBUTiON in Coubse DD. (Compare with Chart 12 by another instructor in the same department.) 73 74 MARKING SYSTEM IN THEORY AND PRACTICE The data from the College of Law, Table VI and Courses EE, FF and GG, are derived from one first- year and two second-year courses. In examining TABLE VI. DiSTEIBUTION OF MARKS IN ThEEE COURSES OF THE COLLEGE OP I/AW. Showing the Per Cent, of Students in the Siao Groups. No. of Ex- Course. Marks. Cond. P-60. P. G. B. EE. empt. EB 251 20.3 17.2 33.6 21.8 7.1 ... 7.1 PF 238 19.3 131 37.0 16.4 13.0 1.2 14.2 GG 222 12.1 16.3 36.5 27.9 6.3 .9 7.2 these charts the reader must remember that height of column, not area, is significant. For instance, the large areas on the left, which represent simply "below 60," are here plotted on the same abscissas used in constructing the graphs for other courses in the University. On the different charts P-60 is rep- resented by the range 60-64, while EE is represented by the range 95-100 (not strictly according to the numerical equivalence just quoted). Since no divi- sion is made at 85, it is impossible to compute the frequency of exemption on the same basis as for the other colleges. This frequency has been calculated, however, as if it included the marks E and EE; as if, in other words, it included marks of 90 and over. Naturally, this frequency is small — 7.1 to 14.2 per cent. We may begin with the consideration of Chart 28, the first-year course. The chief feature is the large number of students conditioned (20.3 per cent.), and the small number reaching the two upper grades (7.1 per cent, get E, none gets EE). What is the explanation of this extraordinary situation? From the evidence at our command it appears that the professor in charge is a severe marker, who be- 35— 30- 25— 20- 15- 10 — 5 — 0- COLLEGE OF L/\w zo.'s % 5EL0W Frtss 7.1 X Exempt O 40 45 50 55 60 6S 70 T5 80 65 90 95 lOO CHART 28. DlSTEIBUTION IH CotJBSE EB. 75 76 MAHKINQ SYSTEM IN THEORY AND PRACTICE lieves in conditioning regularly a certain percentage of his class ; that the examination is searching ; and, finally, that the nature of the work is different from what the beginner has encountered elsewhere in his career, whether in high school or college. The ques- tion may at least be raised whether some change ought not to be made in the conditions under which this course is given, so that not so many as one man in five would fail. In Course FF (not shown graphically), a second- year law course, the conditions are practically iden- tical, so far as conditioned students are concerned, and the curves elsewhere are closely similar, save that in this course the frequency of the two higher marks, E and EE, is somewhat increased at the ex- pense of the mark G. In our last graph. Chart 29, we show the distribu- tion of another second-year law course (Course GG), whose examiner is reputed to be the highest marker in the College of Law. About 12.1 per cent, of the class is conditioned, but P remains the most frequent mark. The trouble with these law curves is evident enough. They use a better number of symbols than other colleges in the University, but these symbols are, in our opinion, improperly rated. The most frequent mark is "Pass," which means inferior to the average, as we have seen in our discussion of the theoretical considerations underlying the use of a limited-division marking system. The mark G (good) should be changed to M (medium), or some other symbol of mediocrity, leaving two divisions above for superior and excellent students, and this mark M should be more frequent than P. CoLLESE OF Law i^.i % 6ELOW R\ss 7.Z% Exempt 35 — 25 — zo— 15 — 5- O ■«) ■« Jo 55 eo 65 70 75 80 85 30 95 lOO CHAET 29. Distribution in Coubse GG. 77 CHAPTER V. SUMMARY AND CONCLUSIONS. Our investigation has led us to tlie following con- clusions — some of them confessedly theoretical and deductive, others incontestable inductions from carefully compiled data. 1. The marking system of any institution of learning plays so important a role in the activities of the institution that its machinery, its significance and particularly its reliability is a matter that de- serves and demands patient and impartial study. 2. Marks may be based upon, performance, upon ability, or upon accomplishment. The last named is, save under unusual circumstances, the quality on which the marks should be based. 3. It is highly probable that ability, whether in high school or in college, is distributed in the form of the probability curve. It is at least possible, and we think it very probable that accomplishment, how- ever, is distributed, under conditions commonly pre- vailing in school and college, in the form of a curve skewed toward the upper range. 4. The number of symbols proposed for record- ing degrees of accomplishment ranges from two to one hundred. Every theoretical consideration and many practical considerations favor a five-division system, based in essence upon five qualities of ac- 79 80 MARKING SYSTEM IN THEOEY AND PRACTICE complishment, viz., excellent, superior, medium, in- ferior and very poor (failure). 5. A curve compounded from more than 20,000 marks shows that at Cornell University the 'pat- tern' distribution is that of a curve skewed toward the upper range, with a mode at 75-79, and the aver- age at approximately 75 (60 being the pass-mark). The frequency of deviations above and below the mode decreases regularly on either side, save for a disturbance at the 60-point. This disturbance is caused partially by an effort on the part of some students to do just enough work to pass, but still more by a strong tendency of examiners to advance marks lying between 55 and 59 to 60 or over. 6. The data obtained for 31 individual courses (7430 marks) shows that the marks of members of the instructing staff are strongly affected by a per- sonal equation — so much that typical distributions taken from high markers and from low markers show no similarity whatsoever. a. The percentage of students obtaining 85 or over (a range which, in many classes, entitles the student to exemption from final examination, and which, by assumption, indicates a quality of work superior to that of the medium student) falls to 1.5 per cent, in one class, and rises to 78 per cent, in another class in the University. fe. Students of medium accomplishment (who by definition are relatively like one another in merit) are by some examiners rated between 85 and 94, but by other examiners 60 to 74. Again, these students are by some instructors spread over a range of 30 points, by others limited to a range of 10 points. c. The marks of the same students, continuing the SUMMARY AND CONCLUSIONS 81 same subject, show a different form of distribution when the instructor is changed. d. Distributions which show radical divergencies in form and tendency may be obtained from the records of two teachers engaged in precisely the same work. 7. These and other variations in the assignment of marks need not always be laid at the door of the instructor. We have shown how the same subject, taught to different groups of students, e. g., to arts students and to engineering students, may yield a differently formed curve of distribution. 8. The curves for individual courses are often multimodal. In other words, there are two or more ranges in the marks which occur with a frequency greater than that of the ranges on either side of them. Commonly, these modes are located at three points, viz., 60-64, 75-79 and 85-89. The first of these is due to the tendency indicated above (Conclusion 5) : the second is the normal 'peak* of average ac- complishment; the third is due to a tendency, an- alogous to the first, to increase the number of stu- dents who are exempt from final examination, i. e., to advance marks from 80-84 to 85 or over. 9. There appears to be a tendency for marks in courses in pure science and applied science to con- form more closely to the theoretically presumptive distribution than do marks in other courses. But this generalization is insecure because, after all, we have charted in detail only 34 out of the several hun- dred courses offered in the University. 10. The marking system employed in the College of Law has the merit of using a restricted number of symbols, but it does not conform to the theoretical 82 MARKING SYSTEM IN THEORY AND PBACTICB curve of distribution, nor was it designed with the proper theoretical considerations (discussed in Chapter II). 11. The marking system used by most faculty members for recording the work of graduate stu- dents (two divisions, satisfactory and not satisfac- tory,) is not to be recommended for use with under- graduates, at least under the conditions that now prevail. 12. We recommend that every institution of learning, at least every high school and college, adopt a five-division marking system, based upon a distribution which should, in the long run, not de- viate appreciably from the following: Excellent, 3 per cent.; superior, 21 per cent.; medium, 45 per cent.; inferior, 19 per cent; very poor, 12 per cent. For purposes of administration the very poor group may be subdivided so that approximately 11 per cent, shall be conditioned, and 1 per cent, shall fail. This distribution conforms well with, theoretical require- ments, and coincides closely with the present prac- tice of Cornell University, as shown by the tabula- tion of 20,348 marks, drawn from a period of three different years and from 163 courses. It is impor- tant to note that, by this proposed system of mark- ing, the meaning of each mark is exactly defined, and in the only satisfactory way by which a mark can be defined, viz., in terms of the frequency with which it can be secured by students under actual working conditions. 13. Furthermore, as Meyer (9, p. 664) advocates, in order to ensure the working of the system the SUMMARY AND CONCLUSIONS 83 distribution actually given should be tabulated at stated intervals, say biennially, and the distribution should be made public, so that every examiner shall know to what extent he conforms to the principles on which the system is based. BIBLIOGRAPHY. (1) Cattell, J. M. ]<]xaminations, Grades and Credits. Popular Science Monthly, 66 : 1905. p. 367. (2) CoLViN, S. S. Marks and the Marking System as an Incentive to Study. Education, 32 : 1912. May, p. 560. (3) Deaeboen, W."f! School and University Grades, liulletin of University of Wisconsin. 1910, No. 368. (4) FosTEE, W. T. Administration of the College Curriculum. Boston: H. Mifflin Co. 1911. Chap. 13. (5) Hall, W. S. A Guide to the Equitable Grading of Students. School Sciewce and Math., 6: June, 1906. j(D jf'^i^ (6) HuET, E. B. Retardation and the Mental Examination of Re- tarded Children. Journal of Psycho-Asthenics, 15 : Sept. and Dec, 1910, p. 31. (7) JuDD, C. H. On the Comparison of Grading Systems in High Schools and Colleges. School Review, 19 : 1910, p. 460. (S) Meteb, M. The Grading of Students. Science, N. S., 28: 1908, p. 243. (9) Meteb, M. Experiences with the Grading System of the Uni- versity of Missouri. Science, N. S., 33 : 1911, p. 661. (10) Saegent, E. B. Education of Examiners. Nature, 70: 1904, p. 63. (11) Smith, A. G. A Rational College Marking System. Jnurnnl q£_E ^cational P sychology, 2 : 1911, p. 383. (12) Steele, A. G^ Training' Treachers to Grade. Pedagogical Seminary, 18 : 1911, p. 523. (13) Stevens, W. L. American Titles and Distinctions. Popular Science Monthly, 63 : 1903, p. 310. 85 INDEX. Accomplishment, how determined, 10; distribution of, 14 ff., 79. Arts and Sciences, College of, 42 ff., 47, 49 ff., 65 ff. Cattell, J. McK., 5, 14, 19, 27, 30, 31, 32. Charts, general explanation of, 23 ff. Colvin, S. S., 6. Conclusions, 79 ff. Cornell University, marking system in, 15 f., 23; distribution of marks 1902-03, 21 ff.; 1903-04, 21 ff . ; 1910-11, 22 ff., 37 ff. Dearborn, W. F., 5, 14, 19, 27, 31, 32. Distribution of marks, theoretical, 11 ff., 79 ff. ; ideal, 33, 82 ; unimodal, 60 ff. ; multimodal, 62 ff., 81. See high-markers ; low-markers. Elimination, effect on distribution, 12 ff. Foster, W. T., 5. Hall, W. S., 5, 33 ff. High-markers, 42 ff. Judd, C. H., 13. Law, College of, 72 ff., 81 f . Low-markers, 49 ff. Marking system, theory of, 9 ff. ; two division, 16 f ., 82; three division, 17 f. ; four division, 18; five division, 18 f., 30 ff., 79 f., 82; Percentile, 6, 19 f. Mechanical Engineering, College of, 45, 69 f . Missouri, University of, 14, 20. Meyer, M., 5, 6, 7, 13, 14, 19, 20, 27, 30, 31, 32, 82. 87 88 Index. Native ability, distribution of, 11 ff. Northwestern University Medical College, distribu- tion in, 33 ff. Performance, marks based on, 9 f ., 79. Probability curve, 11. Euffner, 47. Science, courses in, 62, 70, 81. Skewed curve, 12 f . ; interpretation of, 31 ff. Smith, A. G., 5, 10 f., 14. Starch, D., 20. Steele, A. G., 27. Tables, I, 22 ; II, 38 ; III, 49 ; IV, 62; V, 65 ; VI, 74. Units, best number of, 16 ff., 79 f . Variations produced by change of instructors, 39 ff., 80 f.