THE UNIVERSITY 
 
 0F ILLINOIS 
 
 LIBRARY 
 
 370 
 106 
 
 no.i4H7 
 
 -| wtrC3,tW!l 
 
A 
 
 The person charging this material is re- 
 sponsible for its return to the library from 
 which it was withdrawn on or before the 
 Latest Date stamped below. 
 
 Theft, mutilation, and underlining of books are reasons 
 for disciplinary action and may result in dismissal from 
 the University. 
 To renew call Telephone Center, 333-8400 
 
 UNIVERSITY OF ILLINOIS LIBRARY AT URBANA-CHAMPAIGN 
 
 OCT 18 
 SEP 27 
 
 A 
 
 
 
 1982 
 1962 
 
 L161— O-1096 
 
BULLETIN NO. 14 
 
 BUREAU OF EDUCATIONAL RESEARCH 
 COLLEGE OF EDUCATION 
 
 THE USE OF DIFFERENT 
 
 TYPES OF THOUGHT QUESTIONS IN 
 
 SECONDARY SCHOOLS AND THEIR 
 
 RELATIVE DIFFICULTY 
 
 FOR STUDENTS 
 
 by 
 
 Walter S. Monroe, 
 Director, Bureau of Educational Research 
 
 and 
 
 Ralph E. Carter 
 
 Associate Professor of Educational Psychology 
 Indiana University, Extension Division 
 
 PRICE 30 CENTS 
 
 PUBLISHED BY THE UNIVERSITY OF ILLINOIS, URBANA 
 
 1923 . v 
 

 — -Z^ £? i 
 
 /v-/y 
 
 PREFACE 
 
 This bulletin is a report of one phase of a larger in- 
 vestigation relating to the study habits of school children. 
 The types of questions asked by teachers of their students 
 are important both because of the mental processes which 
 occur in answering them and because an intimate relation 
 exists between the questions asked and the detailed objectives 
 toward which the students work. 
 
 Mr. Carter was consulted in the preparation of the 
 questionnaire. He rendered valuable assistance in deciding 
 upon the tentative list of types of thought questions. Later, 
 advantage was taken of his presence at the University of Illi- 
 nois during the Summer Session of 1922 to secure assistance in 
 the preparation of this report. He is largely responsible for 
 its general organization. 
 
 Walter S. Monroe, Director. 
 
 February 1, 1923 
 
 539709 
 
Digitized by the Internet Archive 
 
 in 2011 with funding from 
 
 University of Illinois Urbana-Champaign 
 
 http://www.archive.org/details/useofdifferentty14monr 
 
The Use of Different Types of Thought Questions 
 
 in Secondary Schools and Their Relative 
 
 Difficulty for Students 
 
 Recent emphasis upon questioning in instruction. The first 
 quantitative study of the questions which teachers use in the class- 
 room was made by Miss Romiett Stevens. 1 In this analysis of 
 questions a distinction was made between those that required memory 
 only and those that required thought. Miss Stevens' discussion has 
 been widely quoted and has been very influential in causing teachers 
 to give more attention to the types of questions asked of their students. 
 It is altogether likely that this investigation is primarily responsible 
 for a material increase in the percent of thought questions which are 
 asked in our schools today. Many of the most commonly used texts 
 on methods of teaching devote separate space to the topic of question- 
 ing. Its importance has been especially emphasized by Charters, 2 
 Strayer, 3 Parker, 4 and Colvin. 5 
 
 Analysis of thought questions. An examination of typical 
 thought questions from the standpoint of the mental processes which 
 they require in answering reveals certain significant differences. Some 
 questions require the student to "compare" two or more ideas; others 
 ask that he "summarize;" still others demand that he "give reasons 
 why." The probable mental processes occurring in the reflective 
 thinking called for rather than the form of the question or the word- 
 ing of the answer have been made the basis of these types of thought 
 
 1 Stevens, Romiett. "The question as a measure of efficiency in instruction," 
 Teachers College Contributions to Education No. 48. New York: Teachers College, 
 Columbia University, 1912. 
 
 2 Charters, W W. Methods of Teaching. Chicago: Rowe Peterson and Company, 
 1912, p. 444. (Chapter XVII). 
 
 •■•Strayer, G. D. A Brief Course in the Teaching Process. New York: Macmillan 
 Company, 1912, p. 315. (Chapter XI) 
 
 4 Parker, S. C. Methods of Teaching in High Schools. Boston: Ginn and Company, 
 1915, p. XXV, 529. (Chapter XX) 
 
 6 Colvin, S. S. Introduction to. High School Teaching. New York: Macmillan 
 Company, 1917, p. XXI, 451. (Chapter XV) 
 
 [S] 
 
questions. Altho there is doubtless considerable overlapping in the 
 types of mental processes which usually occur in answering such 
 questions, there appears to be sufficient differences to justify the re- 
 cognition of a number of types of questions. In this investigation 
 the following tentative list of types of thought questions was used. 
 A more refined analysis would yield probably some additional types, 
 but, on the other hand, for practical purposes it is possible that some 
 combinations of types might profitably be made. It is recognized 
 that the mental processes of different students doubtless vary in 
 answering the same question. In fact a thought question for one stu- 
 dent may be a memory question for another student, or even for the 
 same student on the following day. 
 
 1. Selective recall— basis given. 
 
 Name the presidents of the United States who had been in military life before 
 their election. 
 
 What do New Zealand and Australia sell in Europe that may interfere with our 
 market? 
 
 2. Evaluating recall — basis given. 
 
 Which do you consider the three most important American inventions in the 
 nineteenth century from the standpoint of expansion and growth of transportation? 
 
 Name the three statesmen who have had the greatest influence on economic 
 legislation in the United States. 
 
 3. Comparison of two things — on a single designated basis. 
 
 Compare Eliot and Thackeray in ability in character delineation. 
 
 Compare the armies of the North and South in the Civil War as to leadership. 
 
 4. Comparison of two things — in general. 
 
 Compare the early settlers of the Massachusetts Colony with those of the Virginia 
 Colony. 
 
 Contrast the life of Silas Marner in Raveloe with his life in Lantern Yard. 
 
 5. Decision — for or against. 
 
 Whom do you admire more, Washington or Lincoln? 
 
 In which in your opinion can you do better, oral or written examinations? 
 
 6. Causes or effects. 
 
 Why has the Senate become a much more powerful body than the House of 
 Representatives? 
 
 What caused Silas Marner to change from what he was in Lantern Yard to what 
 he was in Raveloe? 
 
 7. Explanation of the use or exact meaning of some phrase or state- 
 ment in a passage. 
 
 8. Summary of some unit of the text or of some article read. 
 
 [6] 
 
9. Analysis. (The word itself is seldom involved in the question.) 
 
 What characteristics of Silas Marner make you understand wrwfRavflgejDeople 
 were suspicious of him? ' " R i 
 
 Mention several qualities of leadership. 
 
 10. Statement of relationships. 
 
 Why is a knowledge of Botany helpful in studying agriculture? 
 Tell the relation of exercise to good health. 
 
 11. Illustrations or examples (your own) of principles in science, con- 
 struction in language, etc. 
 
 "12. Classification. (UsuallytheconverseofNo.il). 
 
 What is the principle involved here? What is the construction? To what class 
 or genus does this individual belong? 
 
 13. Application of rules or principles in new situations. 
 
 14. Discussion. 
 
 Discuss the Monroe Doctrine. 
 Discuss early American Literature. 
 
 15. Statement of aim — author's purpose in his selection or organiza- 
 tion of material. 
 
 What was the purpose of introducing this incident? 
 Why did he discuss this before that? 
 
 16. Criticism — as to the adequacy, correctness, or relevancy of a 
 printed statement, or a classmate's answer to a question on the 
 lesson. 
 
 17. Outline. 
 
 18. Reorganization of facts. (A good type of review question to give 
 training in organization.) 
 
 The student is asked for reports where facts from different organizations are 
 arranged on an entirely new basis. 
 
 19. Formulation of new questions — Problems and questions raised. 
 What question came to your mind? 
 
 What else must be known in order to understand the matter under consideration? 
 
 20. New methods of procedure. 
 
 Suggest a plan for proving the truth or falsity of some hypothesis. 
 
 How would you change the plot in order to produce a certain different effect? 
 
 Relation to educational objectives of types of questions asked 
 by teachers. Incidentally it may be noted that the types of questions 
 used both for stimulating and directing the mental activity of the 
 learner and for measuring the results of teaching reflect in a subtle 
 way the educational objectives of a teacher. The objectives thus 
 indicated may not agree with those stated by the teacher but they 
 nevertheless are an index of the objectives toward which the students 
 
 [7] 
 
direct their efforts. In studying, students tend to prepare to answer 
 the kind of questions which they think the teacher will ask. It is 
 altogether likely that the study objectives of students are influenced 
 more by the kind of questions asked than by direct statements of 
 aims in the course. 
 
 Purpose and method of this investigation. The purpose of this 
 study, which is Sub-project II of our investigation of the study 
 habits of high school pupils, was announced as follows: "To deter- 
 mine the extent of the use of different types of thought questions in 
 actual school practise and the relative difficulty of these types for 
 students." The method employed was to submit a questionnaire to a 
 large number of teachers in the seventh and eighth grades and in the 
 high school. In this questionnaire the list of the twenty types of 
 thought questions given on pages 6-7 was reproduced. The teachers 
 were urged to study these types carefully before answering any of the 
 questions. It was pointed out that an attempt to classify a question 
 according to this plan might lead to the conclusion that it is a combina- 
 tion of two or three types. However, in such a case, it is probably 
 true that part of the work of answering has been done by the author 
 in the textbook, leaving to the student only that phase of the question 
 which would definitely classify it as belonging to one of the above 
 types. 
 
 Plan of summarizing data. Usable replies were received from 199 
 teachers representing almost every school subject and all parts of the 
 state. A few of these replies did not give answers to one or more of 
 the questions but they were included in the tabulation for the other 
 questions. By subject the number of replies were as follows: English, 
 41; History, 48; Science, 41; Mathematics, 31; Foreign Language, 12; 
 Geography, 11; Commercial subjects, 6; Agriculture, 4. Taking only 
 the replies from teachers of English, History, and Science there were 
 26 for grades VII and VIII combined; 34 for grades IX and X; and 
 45 for grades XI and XII. It was thought advisable to summarize 
 separately the replies for English, History and Science. Those re- 
 ceived from teachers of all other subjects were grouped together. 
 The replies to Questions I and IV obviously do not lend themselves 
 to statistical treatment. The data yielded by the other three ques- 
 tions are summarized in the following tables. 
 
 Limitations of replies to the questionnaire. Several teachers 
 took occasion to mention difficulties which they experienced in filling 
 out the questionnaire blank. That some difficulties were encountered 
 
 [8] 
 
is not surprising because the concept of different types of thought 
 questions was undoubtedly new to most, if not all, of the teachers. 
 The limitations of space prevented an extensive description of each 
 type. Undoubtedly teachers differed in their interpretations of the 
 various types. Furthermore, in answering Question II they were 
 asked not only to differentiate between thought questions and mem- 
 ory questions but also to classify the thought questions under the 
 various types. In doing this, they had to rely upon their memory of 
 the questions they had asked. For these reasons the replies to Ques- 
 tion II must be thought of as giving only a very rough indication of 
 the practise of teachers with reference to the types of questions which 
 they asked. A few teachers reported that their replies were based on a 
 careful analysis of lists of questions which have been used and pre- 
 served. If these lists were reasonably complete their replies should be 
 more accurate than those received from teachers who relied upon their 
 memory of the questions they had asked. 
 
 It may be pointed out that other methods of ascertaining the 
 relative frequency of use of different types of thought questions would 
 not eliminate all difficulties. For example, if a trained investigator 
 should visit the classes of 199 teachers in various parts of the state a 
 sufficient number of times to obtain a fair sample of the habitual 
 practise and should try to make a .record of all types of questions 
 asked he would encounter difficulties in definitely classifying them. 
 It is not possible to judge accurately of the mental processes stimu- 
 lated by a question unless one is acquainted with the previous ex- 
 periences of the pupils in the field of this question. This has been 
 referred to already in pointing out that what constitutes a thought 
 question for one pupil may be merely a memory question for another. 
 The analysis of stenographic reports of lessons would present certain 
 difficulties for the same reason. The teacher knows better than any 
 one else what acquaintance a student has with a topic and what 
 the textbook states about it. For this reason he has the advantage of 
 a casual observer. 
 
 Additional types of questions mentioned. Question I was asked 
 in order to obtain suggestions for supplementing the list of type 
 questions for further study in this field and also to give each teacher 
 an opportunity to report fully his own practise in the use of different 
 types of thought questions in case he did not consider the list ade- 
 quate. The replies to the questionnaire show that the number of 
 teachers who did not report the use of all of the types given in the list 
 
 [9] 
 
UNIVERSITY OF ILLINOIS 
 BUREAU OF EDUCATIONAL RESEARCH 
 
 COLLEGE OF EDUCATION 
 
 Types of Thought Questions 
 
 Name Address 
 
 School Subject School grade 
 
 I. Write in on the blanks at the bottom of the list below any additional types of 
 thought questions you use to any great extent in the grade and subject you have 
 chosen. (Include these additional types in answering all other questions.) 
 
 II. What percent of all of your questions to the class are of each type listed? (See 
 directions, p. 11). Put the percent of each type in the column on the right. 
 
 Types of Thought Questions 
 
 Percent of all 
 questions 
 
 1. 
 
 2. 
 3. 
 4. 
 
 5. 
 
 6. 
 
 7. 
 
 8. 
 
 9. 
 10. 
 11. 
 12. 
 13. 
 14. 
 15. 
 16. 
 17. 
 18. 
 19. 
 20. 
 21. 
 22. 
 23. 
 24. 
 
 Selective recall — basis given 
 
 Evaluating recall — basis given 
 
 Comparison of two things — on a single designated basis. 
 
 Comparison of two things — in general 
 
 Decision — for or against 
 
 Causes or effects 
 
 Explanation 
 
 Summary 
 
 Analysis 
 
 Statement of relationships , 
 
 Illustration or examples 
 
 Classification 
 
 Application 
 
 Discussion 
 
 Statement of aim 
 
 Criticism 
 
 Outline 
 
 Reorganization of facts 
 
 Formulation of new questions 
 
 New methods of procedure 
 
 III. In the left hand margin make an "X" opposite each of the five types for which 
 students answers are least likely to be satisfactory. (See directions, p. 1 1) 
 
 IV. Choose three of the types you marked with an "X." State for each some of the 
 students' most common faults in procedure in answering it. Mention faults of 
 
 [10] 
 
omisson as well as faults of commission. Mention only faults peculiar to this 
 type of question. Use other side of this sheet if necessary. (See directions below.) 
 
 V. If we consider all unsatisfactory answers made by students in school work, what 
 percent of them, in your opinion, are due primarily to bad habits of procedure 
 
 in answering questions? Give separate replies for memory questions % 
 
 and thought questions % 
 
 The following detailed directions for answering the questionnaire 
 were given: 
 
 Before answering any of the questions, decide what subject and what grade 
 (any from VII to XII) you will have in mind in all of your replies. Answer the questions 
 with reference to only one school subject. Fill in the blanks at the top of the first page 
 of the questionnaire. 
 
 Question I. Examine the list of types given above to see if there are not some 
 other types of thought questions that you use to a considerable extent. Altho the given 
 list may seem long and inclusive at first, it is not complete. 
 
 Question II. Before you put down your estimates for Question II, you should 
 decide what is your proportion of thought questions and of pure memory questions. 
 This question calls for a further analysis of your thought questions. It may be advis- 
 able to make rough estimates for all before you fill in the data on the questionnaire. 
 After you have the differences between the types in mind, make the best estimate you 
 can, even tho you may not be very sure in many cases. You are merely declaring your 
 best judgment, not guaranteeing that it is infallible. We want your opinion of what 
 your practise is, not oj what you think it ought to be. 
 
 Question HI. In Question III, we are concerned with the process of answering 
 so we must assume that the student has the information necessary for a satisfactory 
 answer if he will only use it as the type of the question requires. The common starting 
 point for an informal diagnosis of a student's study faults is in his unsatisfactory 
 answers. An unsatisfactory answer may indicate lack of information or certain faulty 
 habits in answering questions. There are many cases in which teachers find out by 
 other means that the student has enough information to give a satisfactory answer 
 but has failed because he did not take the necessary steps and precautions in replying 
 to the question. When some students correct examination papers returned to them, 
 they are heard to say, "I knew that, but I didn't think of it," or "I knew that but I 
 didn't know that was what the question meant." Some of the faults in procedure are 
 common to all kinds of questions; others are peculiar to particular types. In Question 
 III, you are asked to decide on the difficulty of the types for students. In other words, 
 in which type do they have the worst habits of procedure? 
 
 Question IV. It will be helpful in answering Question IV to think of some par- 
 ticular questions of the type under consideration and then express the students' faults 
 in fairly general terms. 
 
 Question V. As long as the unsatisfactory answer is a resultant of poor methods 
 of preparation and poor habits of answering questions, we must correct the latter or 
 take it into consideration in inferring what methods of study are needed. It is important 
 to get an estimate from a large number of teachers of the percent of failures (in answers) 
 that are due primarily to bad habits of procedure in answering questions. 
 
 [11] 
 
is very much greater than the number mentioning additional types 
 Only thirty-nine teachers (19.6 percent) indicated that they used all 
 twenty types of questions. Thirteen teachers supplemented the list, 
 each writing in from one to three additional types. These teachers 
 were distributed among the different subjects as follows: seven in 
 History, three in English, two in Science, and one in Mathematics. 
 Only four of those suggesting additional types of questions indicated 
 that they used all twenty of the types given in the questionnaire list. 
 
 Some of the additional types of questions suggested are clearly 
 included in the list given in the questionnaire. The following are 
 typical: (A) "Connections between historical events," Type 4 or 6; 
 (B) "Cause and results," included in Type 6; (C) "Application to the 
 present time," included in Type 13; (D) "Determining the significant 
 word in a statement," included in Type 19. Other suggested types are 
 not so clearly included but several of them under a fairly loose inter- 
 pretation may be placed with the types listed. Illustrations of these 
 are: (A) "Imagining the results if conditions were different," a 
 special case of Type 6; (B) "Questions on appreciation such as, which 
 do you like best?" a fairly common form of Type 2; (C) "Identify a ' 
 known character in a scene in the story when his name is not mention- 
 ed by the author," a special case of Type 12; (D) "What should you 
 judge from these facts?" This last is broad enough to cover many 
 types. Frequently, it would fall under Type 6. There were a few 
 suggestive questions which are not so easily classified under the 20 
 types given. Some of these are: (A) "Trace the development," 
 mentioned by two teachers; (B) "Estimate the importance of"; . 
 and (C) "Why is this statement true?" a very common question in 
 geometry. By a very liberal interpretation these might be put under 
 Types 9, 2, and 6 respectively but it is likely that many teachers 
 would consider them sufficiently different to justify naming them as 
 additional types. 
 
 The answers to the first question indicate that the list of types 
 is reasonably complete. For practical purposes it is probably better 
 to give a rather loose interpretation in classifying special cases than 
 to try to extend the list. Largely for this reason the authors decided 
 to limit the summary of the replies to the other parts of the ques- 
 tionnaire to the original twenty types. 
 
 Frequency of use of different types of questions. Question II of 
 the questionnaire was asked in order to secure answers for the 
 following: 
 
 [12] 
 
 *^o 
 
1. Which types of questions are most commonly used by teachers 
 in classroom instruction? 
 
 2. Which types of questions are most characteristic of the in- 
 struction in different subjects? 
 
 3. Do teachers in the more advanced grades tend to ask different 
 types of questions from those in the lower grades? 
 
 Table I gives a detailed summary of the replies by forty-eight 
 teachers of History. This table shows that four of these teachers did 
 not report any use of questions requiring selective recall (Type 1); 
 one teacher reported that this type of question formed 1 percent of 
 all his questions; five teachers indicated that this type made up 2 
 percent of all the questions which they asked; eleven named 5 percent 
 as an index of its frequency; eight teachers considered that 15 per- 
 cent of their questions were of this type; and another eight teachers 
 indicated that more than 15 percent of their questions called for 
 selective recall. The median practise is 7.5 percent. 
 
 The outstanding characteristic of the table is the variability of 
 practise which it indicates. In the case of each type of question there 
 were one or more teachers who gave no indication of its use. For 
 several of the types the number of teachers not indicating any use is 
 surprisingly large. Furthermore, there is no type of question which 
 did not receive an indication of at least 5 percent by one teacher. 
 Most of the types received an indication of 10 percent or more. Thus, 
 if we may assume that the replies to this portion of the questionnaire 
 are a reasonably true indication of practise, it is clear that teachers 
 of History vary widely in the types of thought questions which they 
 ask of their pupils and hence necessarily vary widely in the detailed 
 objectives which their pupils strive to attain. 
 
 Similar tabulations were made for English, Science and other 
 subjects combined. The median frequencies for the four groups of 
 subjects are given in Table II. If we consider only English, History 
 and Science, students are most commonly required to answer the 
 following types of thought questions: cause and effect, Type 6; 
 selective recall, Type 1; discussion, Type 14; and evaluating recall, 
 Type 2. The types of questions which are asked least frequently are: 
 formulation of new questions, Type 19; new methods of procedure, 
 Type 20; and reorganization of facts, Type 18. 
 
 Different subjects require different mental processes. An ex- 
 amination of Table II reveals that the frequency of use of some of the 
 
 [13] 
 
OS 
 
 DC 
 
 o 
 
 cr> 
 
 OS 
 to 
 
 U 
 
 < 
 
 w 
 
 H 
 
 oo 
 
 m 
 
 Q 
 
 w 
 
 en 
 < 
 
 O 
 
 M 
 
 H 
 en 
 W 
 
 8 
 
 H 
 DC 
 (J 
 D 
 O 
 DC 
 H 
 
 to 
 O 
 
 C/3 
 
 W 
 
 CL, 
 >H 
 
 H 
 
 
 e 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 r3 
 
 tncOCO w- >sO''fvO lJ "> l '">©©OV©co©'OfNCOOO© 
 
 
 T3 
 
 r-r^-iou->eN©eN»ococO>-i h o « M «i M — 
 
 
 s 
 
 
 I* 
 
 
 
 «^n 
 
 OO CO O CO — ON — 
 
 
 
 
 O 
 
 
 w» 
 
 00*0-— i<N>— i tF cN — i i-i VO <-i i-h 
 
 
 """' 
 
 
 ■* 
 
 ~- ri 
 
 
 *"* 
 
 
 CO 
 
 
 O] 
 
 "■" ' 
 
 
 CN 
 
 co — — -i 
 
 C 
 
 
 
 o 
 
 
 
 
 
 
 CO 
 
 
 
 
 
 
 3 
 
 ^ H 
 
 
 c 
 
 
 
 
 
 4-t 
 
 O 
 
 \0"ivi^MPinONNH CNCN Ot ?i * « h 
 
 X 
 
 
 
 60 
 
 3 
 
 
 
 
 
 O 
 
 
 
 J= 
 
 
 
 H 
 
 
 
 
 00 
 
 « cs * -h n ^^^^Tf — co ~h oi^^h 
 
 >-*- 
 
 
 
 o 
 
 
 
 c 
 
 r^- 
 
 CNCO - N ^h-^ ^hc^^h 
 
 <u 
 
 
 
 
 
 
 Oh 
 
 VO 
 
 CS «S <N i-h Tf-HCN«^ 
 
 >j-> 
 
 — • r~'^Oco© K >O v O vy "> — '•>+'\0 v >0eNN0C4O^-m CO 
 
 
 r-^ ^^ ^— i 1 . 1 *— i 1 
 
 ■* 
 
 CO ■* CN CO CO ■— iCN'-o , ^i-* , CNCO-HTtiTj<Tf coco 
 
 
 eo 
 
 (N •* — CO'* 1 '— i OO h t "t N M ri co-*'t'\ot^CN 
 
 
 cn 
 
 vi'*TfMiflrt(»Noobn\oi s '- 1 vo n "i m n os 
 
 
 ^h^hui^<-h rf< ,— i t-» *f m 'f wi h >o to oo t^ ui 
 
 
 
 rH 
 
 o 
 
 *N , o*0'^lou^^to , 'l»0'*'t , <^ | c , ^^o^ , '-^' 
 
 
 H ^ rt M Ol N CN — ' m « (N 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 to 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 C 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 o • 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 CO 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 to 
 
 
 
 
 
 
 
 
 
 
 
 
 
 to 
 
 
 
 
 
 
 
 
 
 .2 
 
 
 
 
 
 
 
 
 
 
 
 a, 
 
 
 
 
 
 
 2 = 
 
 
 CO 
 
 U 
 
 
 
 
 
 
 
 
 
 £ 
 
 
 
 
 
 
 «s * : 
 
 
 
 
 
 
 
 
 
 
 aj 
 
 
 
 
 
 
 
 o 
 
 CO 
 
 
 
 recall 
 l — sin 
 i— gei 
 
 CO 
 o 
 
 £ r 
 
 
 
 C 
 
 U 
 
 
 
 °C8 
 O 
 
 
 tion o 
 n of n 
 ds... 
 
 
 U 
 
 tive re 
 uating 
 parisoi 
 parisoi 
 
 « o 
 
 ° «« - 
 to C « 
 
 y m £ 
 
 •J 
 > 
 
 tionshi 
 tration 
 sificati 
 ication 
 ussion. 
 ;ment 
 
 
 ganiza 
 lulatio 
 methc 
 
 
 
 coWUUQuW7)<ctisu<Q(flUOei<to2 
 
 
 
 
 
 F<(Nn'i , ' i 'i\oiNOoo\d«rl(i'* l 'isoh'oo'o\0 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 [14] 
 
TABLE II. MEDIAN FREQUENCY OF USE OF DIFFERENT TYPES OF 
 
 THOUGHT QUESTIONS IN ENGLISH, HISTORY, SCIENCE AND 
 
 OTHER SUBJECTS 
 
 Types of Questions 
 
 Median percent of Use 
 
 English 
 
 History 
 
 Science 
 
 Others 
 
 1. Selective recall 
 
 2. Evaluating recall 
 
 3. Comparison — single 
 
 4. Comparison — general 
 
 5. Decision — for or against 
 
 6. Causes or effects 
 
 7. Explanation 
 
 8. Summary 
 
 9. Analysis 
 
 10. Relationships 
 
 11. Illustration or examples 
 
 12. Classification 
 
 13. Application 
 
 14. Discussion 
 
 15. Statement of aim 
 
 16. Criticism 
 
 17. Outline 
 
 18. Reorganization of facts 
 
 19. Formulation of new questions 
 
 20. New methods 
 
 5.4 
 5.9 
 5.4 
 5.0 
 4.4 
 5.8 
 10.0 
 5.6 
 5.7 
 1.8 
 3.6 
 1.6 
 2.9 
 5.7 
 
 7.5 
 
 7.3 
 
 5.3 
 
 5.5 
 
 2.6 
 
 10.4 
 
 2.6 
 
 5.5 
 
 3.5 
 
 3.0 
 
 .96 
 
 .9 
 
 
 
 3 
 
 
 
 6 
 
 2 
 
 3 
 
 2.1 
 
 1.0 
 
 5.3 
 2.8 
 3.9 
 3.1 
 1.1 
 10.2 
 3.3 
 2.9 
 1.8 
 5.7 
 8.6 
 5.2 
 5.1 
 5.3 
 .7 
 2.9 
 
 1.7 
 1.5 
 
 5.1 
 
 .96 
 1.6 
 1.9 
 1.2 
 4.8 
 5.1 
 2.1 
 3.1 
 3.4 
 5.1 
 5.1 
 5.5 
 1.1 
 
 .8 
 3.9 
 
 .8 
 
 .9 
 1.8 
 3.6 
 
 types of thought questions varies in the different subjects. For 
 example, Type 7 "Explanation of the use or exact meaning of some 
 phrase or statement in a passage" is the most frequently used type 
 of question in English but it is eleventh in frequency of use in His- 
 tory. Asking students to give an illustration or example, Type 11, 
 is next to the most frequent type of question in Science but it is sel- 
 dom used in History. Type 10, which asks the student to state a re- 
 lationship, is frequently used in Science but is infrequently asked of 
 students in English. Summaries and outlines are required more 
 frequently in History and English than in the other subjects. Thus, 
 if our assumption that different types of questions require different 
 mental processes is valid, we have here evidence that different types 
 of mental processes are required of students in the different subjects. 
 Hence, we should expect to find that the problem of directing the 
 learning of students is different in different subjects. 
 
 [IS] 
 
TABLE III. PERCENT OF TEACHERS REPORTING NO USE OF CERTAtt 
 TYPES OF THOUGHT QUESTIONS 
 
 Types of Questions 
 
 Grades 
 VII-VIII 
 
 Grades 
 IX-X 
 
 Grades 
 XI-XII 
 
 1. Selective recall 
 
 2. Evaluating recall 
 
 3. Comparison — single 
 
 4. Comparison — general 
 
 5. Decision — for or against 
 
 6. Causes or effects 
 
 7. Explanation 
 
 8. Summary 
 
 9. Analysis 
 
 10. Relationships 
 
 11. Illustration or examples 
 
 12. Classification 
 
 13. Application 
 
 14. Discussion 
 
 1 5. Statement of aim 
 
 16. Criticism 
 
 17. Outline 
 
 18. Reorganization of facts 
 
 19. Formulation of new questions 
 
 20. New methods 
 
 3 
 
 6 
 18 
 
 9 
 21 
 
 9 
 23 
 
 6 
 21 
 35 
 41 
 50 
 46 
 
 9 
 63 
 26 
 26 
 35 
 68 
 44 
 
 18 
 
 29 
 9 
 18 
 32 
 6 
 26 
 23 
 23 
 18 
 26 
 29 
 35 
 15 
 46 
 23 
 21 
 41 
 44 
 53 
 
 11 
 13 
 18 
 13 
 27 
 4 
 18 
 22 
 29 
 20 
 16 
 24 
 22 
 13 
 42 
 24 
 31 
 42 
 44 
 32 
 
 Variations in the use of type questions in different school grades. 
 
 The replies of 113 teachers have been summarized in Table III on the 
 basis of the school grades in which they were teaching. The subjects 
 involved are English, History, Science and Geography. Instead of 
 giving the median frequency of use, the percent of teachers indicating 
 no use of the given type of question has been calculated. Thus, 3 
 percent of the teachers in the seventh and eighth grades reported no 
 use of selective recall. In the ninth and tenth grades 18 percent of the 
 teachers did not use this type and in the eleventh and twelfth grades, 
 11 percent. The general impression prevails that students in ad- 
 vanced classes are asked more difficult types of questions than stu- 
 dents in the lower grades. A comparison of the data for the three 
 grade groups shows that the differences are not very great. In gen- 
 eral, it appears that the variations in the use of thought questions are 
 greater for different subjects than for the different grades. 
 
 The relative difficulty of the different types of questions for 
 students. In Question III teachers were asked to designate the five 
 
 [16] 
 
TABLE IV. TYPES OF QUESTIONS FOR WHICH STUDENTS' ANSWERS 
 ARE LEAST SATISFACTORY 
 
 Types of Questions 
 
 English 
 
 History 
 
 Science 
 
 Others 
 
 Total 
 
 Rank 
 
 1. Selective recall 
 
 2. Evaluating recall 
 
 3. Comparison — single... . 
 
 4. Comparison — general . . 
 
 5. Decision — for or against 
 
 6. Causes or effects 
 
 7. Explanation 
 
 5 
 
 12 
 
 6 
 
 12 
 
 1 
 
 17 
 
 15 
 
 10 
 
 13 
 
 5 
 
 5 
 
 5 
 
 16 
 10 
 15 
 8 
 6 
 12 
 
 9 
 
 6 
 
 10 
 
 12 
 7 
 
 12 
 J9 
 17 
 12 
 14 
 15 
 17 
 4 
 4 
 12 
 14 
 12 
 14. 
 12 
 19 
 
 6 
 
 7 
 
 8 
 
 11 
 
 9 
 
 9 
 
 2 
 
 19 
 
 15 
 
 9 
 
 11 
 
 10 
 
 10 
 
 9 
 
 18 
 
 5 
 
 5 
 
 7 
 
 6 
 
 13 
 
 8 
 
 7 
 
 14 
 10 
 
 5 
 14 
 
 6 
 24 
 25 
 14 
 21 
 20 
 14 
 16 
 29 
 10 
 10 
 10 
 
 9 
 19 
 
 18 
 24 
 
 37 
 45 
 27 
 47 
 18 
 77 
 67 
 47 
 60 
 42 
 33 
 34 
 77 
 39 
 42 
 39 
 33 
 63 
 
 41 
 44 
 
 15 
 8 
 
 19. 
 6 
 
 20 
 1 
 3 
 
 8. Summary 
 
 6 
 
 9. Analysis 
 
 5 
 
 10. Relationships 
 
 10 
 
 11. Illustration or examples 
 
 17 
 16 
 
 13. Application 
 
 1^ 
 13 
 
 14. Discussion 
 
 15. Statement of aim 
 
 16. Criticism 
 
 10 
 13 
 
 17. Outline 
 
 17 
 
 18. Reorganization of facts. 
 
 19. Formulation of new 
 
 questions 
 
 4 
 12 
 
 20. New methods 
 
 9 
 
 
 
 types of questions in which "students' answers were least likely to be 
 satisfactory." A summary of their replies is given in Table IV. The 
 first line of this table should read as follows: Questions calling for 
 selective recall were named among the five most difficult types by five 
 teachers of English, ten teachers of History, eight teachers of Science, 
 and fourteen teachers of other subjects. This type of question ranks 
 fifteenth in the frequency of mention among the five most difficult 
 types. The two types most frequently mentioned were Type 6, 
 causes and effects, and Type 13, application. Both of these were 
 mentioned by seventy-seven teachers or 38.7 percent. Altho we may 
 say that in the judgments of those answering the questionnaire these 
 two types are the most difficult for students, it should be noted that 
 over 60 percent of the teachers did not list them among the five most 
 difficult types. 
 
 Here, as in other tables, the variation in the replies of teachers 
 is very extreme. Even when we consider the teachers of a single 
 subject we find marked variations in their judgments concerning 
 
 [17] 
 
the difficulty of the different types of questions. No type of question 
 is so easy that it is not included among the most difficult five by 
 several teachers. This variation in judgment is probably due largely 
 to the fact that teachers have not recognized the distinction in types 
 of questions which are made here and, furthermore, they have not 
 analyzed the responses of their students in order to ascertain the 
 nature of the difficulties which the students encounter in answering 
 the questions. 
 
 Common faults of procedure in answering different types of 
 questions. In Question IV the teachers were asked to select three of 
 the types which they considered most difficult and to state for each 
 of these some of the students' most common faults in procedure in 
 answering them. An examination of the replies to this question 
 suggests that a considerable number of teachers are satisfied with 
 telling students that the answer is wrong, or what the answer should 
 have been, and that they fail to give much thought to the students' ' 
 faults of procedure in answering the question. It may be pointed out 
 that questions are only a means to an end and this end is the education <| 
 of the student. The answer is in itself relatively unimport- 
 ant. The question fulfils its function only when it stimulates 
 educative processes in the mind of the student. If the mental processes 
 which the question initiates are not educative the asking of the ques- 
 tion has been largely futile. Hence, it becomes highly important for 
 the teacher to give attention to the procedure which the student em- 
 ploys in answering questions in order that the faults of procedure may 
 be corrected so that the student's mental processes will become most 
 effective in his education. 
 
 Of the 199 teachers from whom replies to the questionnaire were re- 
 ceived, twenty-three did not attempt to answer Question IV at all and 
 twenty-four others discussed all three types of questions together or 
 stopped after discussing only one or two. Furthermore, a number of 
 other teachers gave answers which show that they failed to grasp the 
 significance of this question. In substance they said that the difficulty 
 in students' procedure in answering a question is inability to do what 
 is called for. For example, one said that the fault in answering Type 2, 
 evaluating recall, is that "pupils are not always able to evaluate." 
 Another said that in questions on aim "they seem unable to see any 
 motive behind the statement of the author." Still another said that 
 the fault in Type 6, causes and effects, is that "they are unable to see 
 causes and effects." Still others mentioned objective faults in the 
 
 [18] 
 
answers rather than faults in procedure. For example, one teacher 
 mentioned that "minor details are included while leading points 
 are omitted." Another stated that "applications are not as good as 
 could be expected." Still another teacher simply stated that "the 
 answers are inaccurate." General faults, such as "lack of practise in 
 answering questions," "carelessness," "lack of concentration," "pupils 
 do not know how to study," and so on were mentioned by one or more 
 teachers. 
 
 It may be pointed out that teachers who fail to become definitely 
 conscious of the difficulties which their students encounter will be 
 unable to give them much definite constructive assistance in this 
 phase of their learning. Probably the most significant conclusion to 
 be drawn from the answers to this part of the questionnaire is that 
 many teachers are failing to give attention to the procedure which 
 students use in answering questions. They appear to be concerned 
 largely with the accuracy of the answer and when it is wrong they 
 fail to seek the cause in the procedure which the student has used. 
 
 Several of the faults which appear to be suggestive are given 
 below. This list is not a complete statement of the faults of students 
 in answering questions but should prove helpful to teachers who are 
 desirous of ascertaining the reasons why students fail to answer 
 questions satisfactorily. 
 Type 2. 
 
 The pupil fails to see real basis on which evaluation should be made. 
 
 Type 3. 
 
 A student seldom stays "on a single designated basis." 
 Type 3 or 4. 
 
 The pupil begins to answer the question before he has thought out what points 
 should be included in the comparison. He often includes minor comparisons and omits 
 the important points. 
 
 Unless there is a single designated basis for comparison, pupils fail to see all of the 
 possibilities of comparison. The slothful pupil is content with one or two obvious com- 
 parisons. 
 
 The pupil can state some information about each thing compared, but omits 
 the comparative connection between the two things. The pupil gives comments on 
 one thing and omits the discussion on the other thing, showing how the two are alike 
 comparatively. 
 
 In making comparisons, students usually give characteristics of things to be 
 compared, or describe them separately but fail to make a complete comparison. 
 Type 4. 
 
 In comparing two things in general the worst fault is in the students' forgetting 
 some of the important points to be compared. Often enough suggestions must be 
 given so that the question finally should be placed under Type 3. 
 
 [19] 
 
Pupils frequently fail to distinguish the important from the trival. The pupil' 
 prejudices — likes and dislikes — influence the comparison. 
 
 Type 6. 
 
 Students confuse cause and effect. Method requires real thinking and reasoning — 
 the hardest thing any student can be asked to do. 
 
 Students often know a statement is true but on account of lack of thoroness can 
 not give causes or reasons. 
 
 Type 6 or 7. 
 
 Pupils, when asked, "Why?" seem to forget all reasoning and usually state the 
 first thing that comes to mind and fail also to go far enough in explanation. 
 
 Type 7. 
 
 Pupils have difficulty in finding words to express the thought without repeating 
 word for word parts of the passage to be explained. The chief difficulty seems to be the 
 limitations of their vocabulary. 
 
 The trouble here is that pupils seem to have failed to study the statement under 
 question sufficiently and so have quite an indefinite notion of the words used. And too, 
 even if they succeed in getting a fairly good understanding of the text read, they are 
 too careless of the choice of their own words in reproducing the meaning exactly. 
 They are too hasty in the study of the passage and have not yet been trained to go 
 slowly and get facts one at a time. 
 
 Type 8. 
 
 A discussion, like a summary often lacks point. The student fails to organize 
 material and grasp essentials. 
 
 Students fail to properly organize data mentally before expressing them. 
 
 Type 9. 
 
 Here pupils are often prone to feel satisfied if only a few of the factors or qualities 
 involved in the analysis are learned and reproduced. Many times but one factor will 
 be offered when to completely analyze the problem will require several. Being allowed 
 to "get off" with a minimum of thinking is one of the chief causes of poor analysis. 
 
 Type 10. 
 
 Pupils often fail to get the facts correlated before they attempt to state the exist- 
 ing relationships. 
 
 Pupils do not stop to analyze in order to see the relationship but answer without 
 giving much thought. 
 
 Pupils answer various separate facts instead of showing relationship. These facts 
 are usually true and have a bearing upon the question but the actual relationship is 
 usually missing in the answer. That is, the pupil does not seem to see how a change in 
 one fact would influence some other related fact. 
 
 Type 11. 
 
 Illustrations or examples. This is not so difficult after the student has had some 
 practise, but at first he always tries to find his illustrations in some field remote from 
 his own. It takes him a while to realize that scores of illustrations come daily under his 
 observation — ;/ he observes. 
 
 [20] 
 
Type 12. 
 
 In classifying, the pupils make their answers too readily without first trying one 
 class then another or without thinking whether the one they choose actually does fit 
 better and why. 
 
 The student's failure in Type 12 is due mainly to his failure in Type 7. He fails 
 to decide correctly upon a construction because he has not thought out carefully the 
 exact meaning of the sentence. This in turn is due to mental laziness or carelessness. 
 
 Type 14. 
 
 He discusses each phase at randcm and separately without relating it to others. 
 A discussion like a summary often lacks point. The student fails to organize 
 material and grasp essentials. 
 
 Students fail to properly organize data mentally before expressing them. 
 Students do not group their facts together sufficiently to discuss them properly. 
 
 Type 17. 
 
 A pupil doesn't really outline into main facts with heads under them, but makes 
 long statements one after another, or puts steps in before their time. 
 
 General. 
 
 Pupils will not read and get a thoro knowledge of the subject-matter first. 
 
 With Types 8, 14, and 17 pupils do not summarize or discuss or outline in a way 
 to make clear to another the things under discussion. They are not logical in the ar- 
 rangement of their ideas. They answer, at least 85 percent of them, only well enough 
 so that one who knows already can tell that they know, but only about 15 percent 
 can arrange and subordinate well enough to be clear to the unintiated. 
 
 A common fault is careless reading of the question (or hearing it as the case 
 may be). This is especially true in Algebra. Another fault is "saying the first thing that 
 comes into mind" instead of carefully considering its application to the problem under 
 discussion. In Geometry, for example, the student does not rapidly recall all theorems 
 pertaining to the subject in hand, eliminate those inapplicable, and so discover the 
 correct one. 
 
 Percent of unsatisfactory answers due to faulty procedure in 
 answering questions. In Question V the teachers were requested to 
 indicate separately for memory questions and for thought questions 
 the percent of unsatisfactory answers which, in their opinion, were 
 due primarily to bad habits of procedure. A summary of their re- 
 plies is given in Table V. As previously indicated it appears certain 
 that a considerable number of teachers have given relatively little 
 thought to the faults in the procedure used by students in answering 
 questions. Consequently, the replies to this question must be con- 
 sidered as representing no more than rough estimates. In the case 
 of thought questions the median percent of unsatisfactory answers 
 due to faulty procedure is approximately 50 for all of the subjects. 
 In the case of memory questions faulty procedure is considered to be 
 a less potent cause. There are two factors that probably tend to 
 
 [21] 
 
C/5 
 
 z 
 
 o 
 
 H 
 in 
 
 W 
 
 P 
 
 c 
 
 o 
 z 
 
 5 
 w 
 
 Z 
 
 < 
 
 »— ( 
 
 W 
 
 OS 
 D 
 Q 
 W 
 U 
 
 OS § 
 
 — "B 
 
 D e 
 
 < l " H 
 
 fa o 
 
 o -* 
 
 r* « 
 
 D -5 
 Q $ 
 
 % •* 
 
 — V 
 
 fa a 
 
 £ £ 
 
 Z o 
 
 < U 
 
 ^ < 
 
 OS w 
 
 o 
 
 H 
 U 
 
 :- 
 < 
 
 fa 
 
 o 
 
 I-H 
 
 z 
 w 
 u 
 
 os 
 w 
 
 Oh 
 
 w 
 
 PQ 
 
 < 
 :- 
 
 ■- 
 
 -H ^- r< — Tf fSJ 
 
 — (N « 
 
 p^_v^3^H\coJr^--* 
 
 io — ~ 
 
 - m « oo n 
 
 ~ W) 
 
 t "i c t n 
 
 « fV) ^ (Vj -^« 
 
 vo r~ — < o) — 
 
 W 
 
 cso , oor--\0'* < r^ooo 
 
 — i {N -* 
 
 •^ocr^r-.^or^co— to 
 
 n "i n "i 
 
 C-l — ' <N c*5 
 
 W 
 
 — — n i^ 
 
 r«- <-> <n r~- cj 
 
 
 
 [22] 
 
make these estimates lower than they should be. Many teachers 
 showed by their replies to Question IV that they had very little ex- 
 perience in analyzing answers for the purpose of determining the cause 
 of faulty answers. Naturally they could not be expected to recognize 
 the real extent of such faults of procedure on the part of the members 
 of their classes. Other teachers use such a limited range in types of 
 questions that they would not have a chance to observe as many 
 faults in answer technique as teachers who use a greater variety of 
 types of questions. 
 
 Relation between frequency of use and difficulty. There seems 
 to be very little relation between the difficulty of a type and the 
 frequency of its use. Type 6, causes and effects, is the most commonly 
 used type and at the same time it, together with Type 13, application, 
 is considered the most difficult for students. Only one of the five 
 types reported as most difficult, Type 18, reorganization of facts, 
 is among the five least frequently used; only one of the five types 
 reported as least difficult, Type^S^comparison on a single basis, is 
 among the five most frequently used. S£-»_^x-* (^ ^jf 1 * ^ 
 
 Relation of frequency of use of types of questions to emphasis 
 upon different types of learning. An earlier bulletin 1 of the Bureau 
 of Educational Research reports the relative emphasis on different 
 types of textbook study- in the various subjects. We now have some 
 data on the question, "How does the frequency of use of types of 
 questions correspond with the emphasis on the different types of text- 
 book study required of students?" Some of the types of study 
 recognized in the report are too broad to be tested by any one type 
 of thought question. There are, however, question types that cor- 
 respond rather closely with certain of the study types. 
 
 To get a rough measure of the relation of the frequency of use of 
 the different types of thought questions to the emphasis upon the 
 types of learning, we may arrange the types in order of frequency of 
 use and divide the list into quartiles (four equal parts). For instance, 
 in English, Study Type II is in the first quartile (i. e. the three most 
 frequently required study types); Study Type X is in the fourth 
 
 Monroe, Walter S. "Types of learning required of pupils in the seventh and 
 eighth grades and in the high school." University of Illinois Bulletin, Vol. 19, No. 15, 
 Bureau of Educational Research Bulletin, No. 7, Urbana: University of Illinois, 1921, 
 16. p. 
 
 [23] 
 
Types of Textbook Study 
 
 Types of Questions 
 
 II. Summary of central ideas in the 
 lesson studied. 
 
 III. Prepare an outline of principal points 
 and supporting details in the lesson 
 arranged to show order of relative 
 importance and relationship. 
 
 VI. Discovery of collateral or illustrative 
 material for topics or problem under 
 discussion. 
 
 VIII. Appreciation of the significance of 
 each word used in a concisely ex- 
 pressed statement or principle. 
 
 IX. A clear comprehension of the essen- 
 tial conditions of a problem which is 
 to be solved. 
 
 X. Discovery of new or supplementary 
 problems related to the topic being 
 studied. 
 
 8. Summary, 
 
 17. Outline. 
 
 11. Illustration or examples. 
 
 7. Explanation of the use or meaning c 
 some phrase or statement in a passagt 
 
 9. Analysis. 
 
 19. Formulation of new questions. 
 
 quartile (i. e. one of the least frequently required study types). 7 
 Table VI shows the extent of agreement in rank in frequency of use 
 of some corresponding types of study and questions. The first column 
 gives by number the corresponding types as listed above. The first' 
 number given in the columns for the separate subjects designates the 
 quartile rank of the study type on the same horizontal line; the, 
 second number designates the rank of the corresponding type of 
 question. It will be noted that the rank agrees in seven out of eighteen 
 possible cases. In English the disagreement is greatest in the case of 
 questions calling for analysis (first rank) and study requirements of a 
 clear comprehension of the conditions of a problem (fourth rank); 
 in History, the discovery of collateral or illustrative material ranks 
 second in study requirements but questions calling for illustrations or 
 examples rank fourth in use; in Science, the same study requirement 
 ranks fourth and the corresponding question type ranks first. Thirty- 
 three percent of the Physics and Chemistry teachers in the first ques- 
 tionnaire mentioned Study Type X, discovery of new or supple- 
 mentary problems, as a major type of study in these subjects, yet 
 44 percent of the teachers of these subjects by their admission do not 
 ask any questions of Type 19, formulation of new questions, to their 
 
 'These facts are derived from Table II of the Bulletin referred to above. 
 
 [24] 
 
CO 
 
 o 
 
 H 
 
 CD 
 
 C 
 
 O 
 
 co 
 W 
 O, 
 >« 
 H 
 
 Q 
 < 
 
 Q 
 D 
 H 
 
 co 
 
 O 
 
 co 
 W 
 
 CL, 
 tH 
 
 H 
 fa 
 O 
 
 u 
 
 Q 
 Z 
 
 2 
 
 co 
 
 £* 
 a! 
 O 
 U 
 
 >" 
 
 l 
 
 
 
 
 3 
 
 .- to 
 
 Q 
 
 co I 
 
 £ 
 
 + + 
 
 Jm 
 
 .3 2? 
 
 d 
 
 
 < 
 
 I W 
 
 !> 
 
 
 
 
 j 
 
 
 
 « ._; >-3 
 
 >< 
 
 X CO oo 
 
 < 
 
 ^+ 4- + 
 
 > 
 
 60 Ml 60 ti 
 
 
 
 c c c .2 
 
 
 
 UJ UJ UJ I 
 
 
 
 
 w 
 
 '0 
 
 u 
 
 co 
 
 O 
 < 
 
 + 
 
 *J . 60 £ .60 
 
 
 *^ .j* 
 
 
 u 
 
 
 CO ■* •— i CN CO CO 
 
 o 
 
 
 
 •z. 
 
 w 
 
 
 
 
 
 
 
 
 CJ 
 
 
 
 co 
 
 .£•-* 
 
 
 
 -a c 
 
 CO •* •* CO CO CN 
 
 
 
 
 *3 _k: 
 
 
 
 1 s 
 
 CN CS Tf CO CN ■*■ 
 
 t- 
 
 C/X 
 
 
 a: 
 
 
 
 o 
 
 
 
 
 
 
 M 
 
 
 
 ffi 
 
 £••* 
 
 
 
 3 rt 
 
 CN -h cN co if co 
 
 
 co* 
 
 
 
 « ^i 
 
 
 
 | s 
 
 CN CO CO — i r-> Tf 
 
 s 
 
 a* 
 
 
 c/3 
 
 
 
 
 
 
 O 
 
 
 
 
 
 2: 
 
 
 
 w 
 
 -£■ ■* 
 
 
 
 ^3 C 
 3 P3 
 
 — CN CO CN ■*■ «*• 
 
 
 
 
 CO — 
 
 
 o 
 z 
 
 •Si 
 
 1) 
 
 co f^ — i t^- Cs OS 
 
 
 3 
 
 
 
 C 
 
 
 o a 
 
 
 
 ^ - 
 
 
 
 
 
 
 ^_ 
 
 
 a; 
 
 
 
 o 
 
 co 
 
 — — •> 
 
 
 
 [25] 
 
high school juniors and seniors. It would seem from the data that 
 some teachers may not be using the types of questions that would 
 stimulate and test students' achievements in some of the types of 
 study they are expected to use. 
 
 Conclusions. Probably the most significant conclusions to be 
 drawn from this investigation are: (1) teachers are not sufficiently 
 conscious of the types of questions which they are accustomed to ask 
 and of the significance of these types, and (2) in general teachers do 
 not analyze unsatisfactory answers to questions in order to ascertain 
 whether such answers are due to a faulty technique on the part of 
 the student. 
 
 A number of other conclusions are worthy of mention. (1) 
 Teachers of the same subject vary widely in the extent of their use of 
 different types of thought questions. (2) The frequency of the use of 
 a given type does not seem to depend very much on (a) the school 
 grade, (b) the subject, or (c) the supposed difficulty of the type. It is 
 very likely that some teachers, who expect their students to use cer- 
 tain types of study, do not use the types of questions that are best 
 suited to test their students' achievements due to those particular 
 types of study. (3) IfTeachers individually think that certain types of 
 questions are more difficult for students than others. However, there 
 is no very great agreement among teachers as to the relative diffi- 
 culty of the various types.jj 
 
 [26] 
 
BULLETIN NO. 15 
 
 BUREAU OF EDUCATIONAL RESEARCH 
 COLLEGE OF EDUCATION 
 
 THE CONSTANT AND VARIABLE ERRORS 
 OF EDUCATIONAL MEASUREMENTS 
 
 by 
 
 Walter S. Monroe 
 Director, Bureau of Educational Research 
 
 PRICE 25 CENTS 
 
 PUBLISHED BY THE UNIVERSITY OF ILLINOIS, URBANA 
 
 1923