How to Evaluate an Instructional Program

Evaluation in Education Student Assignments We welcome your comments and suggestions for problems and modifications for each group, so as to provide the authors with suggestions for reflection and to promote **** learning and improvement. How to Conduct Effective Classroom Observation Author: Ji Shiwei 2007101344 Zheng Bixia 2007101308 Zheng Peis 2007101311 Zheng Weixin 2007101346 Zhong Mingjun 2007101302 Department: Chinese Language and Literature of Chinese Language and Culture Abstract: The classroom is the key to the implementation of the curriculum reform, and classroom observation is precisely one of the effective and operable methods for studying classroom teaching. Classroom observation is a practical and effective way to study classroom teaching. Classroom observation is one of the effective ways to study classroom teaching. It is the key and core issue of the current teaching research to study the classroom teaching level in depth with the help of classroom observation. Classroom observation is one of the most frequent and basic methods of classroom teaching research, an important method suitable for studying classroom situations, an important way to promote teachers' professional development, and a basis for educational decision-making and evaluation. This paper briefly describes the meaning, methods and significance of classroom observation, and then explains the importance of effective classroom observation for teaching. Keywords: classroom observation, types, tools and methods 1 Part I Preface Classroom observation has been practiced since the beginning of classroom teaching. However, classroom observation, as a scientific research method, is still "an omitted professional competence of teachers". Although classroom observation has been introduced in China for a long time, some schools or teachers have also conducted some research on classroom observation. However, as an experience from other countries, how can it take root and develop in China? There are still many theoretical and practical issues that deserve more in-depth study. For example, can classroom observation contribute to the improvement of classroom learning? Can classroom observation promote teachers' professional development? How can foreign classroom observation techniques be improved to fit Chinese classrooms? How to train teachers with a set of skills to become qualified observers? We, as a small group of university students, have explored some of the above questions in theory and practice after a month's continuous research, and have made creative explorations in some aspects. We are willing to share them with you here, and also willing to cooperate with colleagues who are interested in them to continue to promote the study of Chinese-style classroom observation. Classroom observation is a professional activity of recording, analyzing and researching the classroom situation through observation, and on the basis of which it seeks to improve students' learning in the classroom and promote teachers' development. It requires the observer to collect information directly (or indirectly) from the classroom with a clear purpose, using his/her own senses and relevant auxiliary tools (observation form, audio-visual equipment), and to analyze and study the information accordingly. It is] an important part of teachers' professional learning. （Classroom observation can be categorized according to different criteria: 1. According to the way the information is collected and the attributes of the information itself, it can be divided into quantitative and qualitative observation. 2. According to the relationship between the observer and the classroom under observation, it can be divided into self-observation and classroom observation of other people. According to the cooperative relationship between the observers, it can be divided into cooperative classroom observation and independent classroom observation. According to the selection of observation objects or contents, it can be divided into centralized observation and decentralized observation. 5、According to the different purposes and roles of observation, it can be divided into diagnostic observation, refining observation and thematic observation. Many classroom observation classifications are relative, and they often cross and overlap, with multiple attributes. Teachers should treat them flexibly and apply them comprehensively. （Classroom observation is a paradigm for listening to and evaluating lessons, and a way or method for teachers to study the classroom, but it is not the only way or method. It is not the entirety of teachers' teaching and research activities, and it is not a panacea for all ills; it can only solve the problems that it can solve. If teaching is an art, classroom observation is a scientific approach to solving some of the problems in the art of teaching. It is good at analyzing and diagnosing classroom behaviors locally, but not at grasping classroom events holistically, comprehensively, and macroscopically. It is good at analyzing and diagnosing classroom behaviors partially, but not good at grasping classroom events as a whole, comprehensively, and macroscopically. It mainly accomplishes three tasks: 3) describing teaching and learning behaviors and diagnosing teaching problems, 2) helping teachers to improve specific teaching problems in the classroom, and 3) changing the daily classroom research behavior of teachers. However, the limitations of classroom observation are obvious, mainly in the following ways: it can only observe visual and perceptible intuitive phenomena and behaviors, such as teacher's words, students' classroom responses, classroom activities, etc., and it is a kind of induction method to understand the essence from the phenomena; it can only select one or several observation points or classroom behaviors to conduct a detailed and in-depth observation and research, and it can not embrace all the behaviors or a larger number of observation points. It requires the observer to receive certain professional training and possess appropriate observation skills, and it requires the observer to be able to concentrate on observation, collect relevant information in a timely and accurate manner, and make decisions at any time. It requires the observer to come to the lesson with the mindset of a "home lesson," and it requires the observer to be willing to be observed by others and not be influenced by on-site observations. It requires a certain amount of time, equipment, and technology to complete the three steps of the program; it also requires a cooperative ****-symbiotic matrix in which teachers can engage in free, shared, and reciprocal dialogue and communication to ensure the continuity and effectiveness of the observation. (Zhu Weiqiang, Zheng Donghui) Part III: The Process and Main Methods of Classroom Observation (a) Basic Steps of Classroom Observation 1. Phase 1: Preparation for Classroom Observation: Determine the purpose and planning of the observation. Firstly, the time, place and number of observation should be determined. Second, determine the center or focus of the observation according to the purpose of the study. Thirdly, design or choose the way or tool to record the observation. 4 2. Stage 2: Classroom Observation: Entering the Classroom and Recording Information The implementation stage of classroom observation involves entering the research situation and recording the required information in the research situation according to the pre-planned and chosen recording methods. The observer should ask for consent and establish a relationship of trust between the observer and the observed as soon as possible, in order to eliminate suspicion and wariness on the part of the observer and curiosity on the part of the students. Through different ways of recording classroom observation, the observer records the pattern of the observed behaviors, including the time of occurrence of the behaviors, the frequency of occurrence, the content and form of verbal or non-verbal activities of the teachers and students, and also includes some textual descriptions of other behaviors of the observed subjects, as well as the observer's feelings and understanding of the scene, and some audio-visual materials. Stage 3: Work after classroom observation: analysis of data and presentation of results The data collected through classroom observation are generally qualitative and quantitative. The ultimate purpose of classroom observation is not to write and publish a research report or a paper, or to prove, fill in or construct a theory, but more importantly to promote teaching and improve practice. （Classroom Observation Framework The Classroom Observation Framework breaks down the classroom into four dimensions: student learning (preparation, listening, interaction, autonomy, and five perspectives), teacher teaching, nature of the curriculum, and classroom culture, with each dimension consisting of five perspectives, and each perspective consisting of three to five points of view, for a total of 68 points. First, the Classroom Observation Framework provides a framework for teachers to understand the classroom. The Observation Points of the Framework support teachers in thinking about the classroom in terms of "points" (what is the classroom?), while the 68 points support teachers in thinking about the classroom in terms of "points". The 68 points, 20 perspectives, and 4 dimensions of the framework support teachers' understanding of the classroom in terms of its "surface", so the framework leads teachers to understand the classroom in terms of both points and surfaces. Therefore, the Classroom Observation Framework leads teachers to understand, reflect on and improve the classroom from both the "point" and "surface" perspectives, so as to enhance the effectiveness of teachers' teaching and the quality of their professional development. Secondly, the framework provides a reference system for teachers to select observation points and develop observation tools. When carrying out classroom observation, teachers, after familiarizing themselves with classroom observation in the light of their own teaching practice and according to their own developmental needs, look for relevant observation dimensions, perspectives and points in the observation framework and design their own observation points. After determining the observation points, they can choose or design observation tools, such as scales, record sheets, questionnaires, etc., according to the structure of the observation framework. They can also choose partners for cooperative observation, discuss the contents of the observation for the division of labor, and discuss the rules of observation for both parties. （The classroom is so complex and ever-changing that it is impossible to observe everything that happens in the classroom; but if we don't know what we are looking for, we won't be able to see more. Therefore, classroom observation requires that the point of observation be determined in advance based on the quality of the observation site, the purpose of the observation, and the content of the observation. First of all, the observation point should be determined according to the quality of the observation point - observable, recordable and interpretable. This is determined by the characteristics of observation. We can only observe specific behaviors (e.g., questioning and responding, explaining and distinguishing, coaching and practicing, teacher movement and use of instructional tools, etc.), but it is very difficult to observe what is going on inside the head of the teacher and student. Second, the point of observation should be tailored to the needs of both the observer and the individual being observed. Teachers at different stages of development have different concerns (they can identify observation points according to the areas of teaching and learning that they need to strengthen, and use this as a reference point for their own improvement). Teachers may, in fact, find that the needs of the observer and the observed are often not the same, and this needs to be negotiated during the pre-sessional meeting. Again, it is important to determine the point of observation based on the needs of the collaborative. One of the prerequisites for the formation of a classroom observation partnership is that it has a common goal, whether it is to study a topic, to develop a teaching style, or to improve some aspect of classroom teaching. Therefore, when determining the observation points, we should also consider the needs of the collaborative, on the basis of which we should "design-observe-reflect-improve", so as to form a teaching and research team. On this basis, "design-observation-reflection-improvement" will form a follow-up chain of teaching and research activities. （There are many ways to record classroom observation, and you should choose the way you are good at recording observation according to the specific content and type of observation. Generally speaking, classroom observation recording methods can be divided into qualitative recording methods and quantitative recording methods. Quantitative recording is to deconstruct and categorize the elements in the classroom in advance, and then record the behaviors in the categories in a specific period of time. The main types of records are rating scales and classification systems. Qualitative recording methods are non-numerical presentations of observation content, including: (1) description system, i.e., various forms of description other than numerical description of the observation target under a certain classification framework, which is a kind of quasi-structural recording method of qualitative observation; (2) narrative system, i.e., without pre-set classifications, to make a detailed and authentic written record of observed events and behaviors, and also to carry out on-site subjective evaluation. subjective evaluation. (3) Graphic recording, i.e., directly presenting relevant information in the form of location and environment maps. (4) Technical recording, i.e., the use of audio and video tapes, photographs, and other electronic forms to make on-site permanent records of the behavioral events to be studied. Quantitative and qualitative recording methods can be used to complement each other. The data and information obtained should reflect the real teaching environment and classroom activities as much as possible. How to choose the existing observation and recording tools Three main factors to consider are: first, the observation point; second, the observer's own characteristics; and third, the observation conditions. 7 3. Why we need to develop new tools for recording observations The existing mature tools for recording observations have been tested in practice in terms of logical rigor and scientificity, but their limitations are also obvious, so the independent development of tools for recording observations has become a more realistic choice. How to develop new observation recording tools There are mainly three stages: First, the analysis and design stage, firstly, we should analyze the elements of the observation object (content) and the characteristics of the observation class, and then design the observation recording tools according to the specific situation of the observation class. The second stage is the trial and correction stage. After the observation tool is released, it must be tested for its scientific validity. Generally speaking, observation tools developed by teachers themselves are easy to use and can be explained in a self-explanatory way. Although there may be problems in one way or another, the process of developing the tools can improve the teachers' theoretical quality, design ability and the standard of collaborative research. (v) Main methods of classroom observation 1. Drawing up an outline for observation Focus of observation: introduction of questions (introduction to the new lesson) Outline of observation: (1) Is the space for questions appropriate? (2) Is the question conducive to mobilizing students' initiative? (3) Are the questions directed in a certain way? (4) How operational are the questions? Teacher A: Teachers: Please give examples of parallelograms in your life. Teacher A: Teacher: Please give examples of parallelograms in your life. Students: classroom doors, classroom desks, classroom windows, etc. (Students respond enthusiastically.) 8 Teacher: Good! These are all parallelograms. Today we will study the properties of parallelograms. （Teacher B: What are the properties of parallelograms? （Teacher B: What is the difference between the following two quadrilaterals? Students: ① is an irregular quadrilateral and ② is a parallelogram. （Teacher B: What is the difference between the following two quadrilaterals? （Who remembers the definition of a parallelogram? （Who remembers the definition of a parallelogram? (Enter the new lesson.) Students: (A few raise their hands.) Teacher C: Cut out a pair of equal triangles and try to see what kind of geometric shapes can be made from them. Students: Cut out pairs of congruent triangles of different shapes and then put them together. （Teacher: Please go to the blackboard to show your puzzles. Students: (students enthusiastically) show the following pictures: ① ② ③ 9 Teacher: Who has a different approach? (Focus on students' individual development, reflecting the democracy of teaching) Student: Students continue to do the jigsaw puzzles. (The focus of teaching seems to be on puzzles, not on parallelograms.) Teacher: Let's continue. From the three teachers' teaching introductions, we can get a glimpse of the teachers' teaching concepts and their use and understanding of the teaching materials, as well as a direct observation of the quality of the questions set by the teachers with the help of the question introductions. Teacher A's questions were student-centered, with some room for independent thinking, but they also served the purpose of recalling the concept of parallelograms. It is natural for students to recall the shape of a parallelogram, and even the concept, by giving examples. Modern psychological research has shown that self-examples reflect learners' level of conceptual understanding. Teacher B's questions were intuitive, simple and to the point (the questions were clearly directed). However, there was little room for questions, and students were at the level of memorization, which was not easy to mobilize students' initiative and interest in learning. Teacher C's questions were directly based on the scenarios in the textbook, and were highly actionable. Students were eager to see their own work, which made it easy to mobilize students' learning initiative. Although the questions were not clearly directed, they had a positive transfer to the study of the properties of parallelograms. Unfortunately, the teacher's guidance was far from the main theme of the lesson (of course, it was not pointless for students to explore what other shapes they could put together, but classroom time was limited). The teaching materials for this lesson are written with two objectives in terms of knowledge: one is to make students recall the concept of parallelogram through the jigsaw puzzle activity, and the other is to gradually transition from this activity to the study of the nature of parallelograms. At the same time, we pay attention to the accumulation of students' activity experience and the mobilization of learning emotions. It is worthwhile to advocate the introduction of problems in teaching and even the creative use of teaching materials, but it is better not to deviate from the design intention of the "original" teaching materials. 2. Use of scales for observation We can observe the classroom with the help of observation scales that have been developed by others. The following is the Flanders Classification of Teacher-Student Interaction (FICA): Flanders Classification of Teacher-Student Interaction (FICA): ①Accepting students' feelings ②Praising or encouraging students' behaviors Indirect Influence ③Accepting students' ideas and opinions Teacher speaking ④Questioning students ⑤Explaining Direct Influence ⑥Giving guidance or instructions ⑦Criticizing or defending authoritative ⑧Students' passive speech (e.g., answering the teacher's questions) Students speaking ⑨Students speaking on their own initiative or questioning the teacher Still Silence, doubt, or temporary pause Autonomous Neutral Response Autonomous Neutral Response The Flanders Classification of Teacher-Student Conversational Interactions divides teacher-student conversational activities into ten categories, each of which has a code (i.e., a number that indicates the type of behavior). Using the Flanders Classification of Teacher-Student Conversations, we can analyze the frequency of teacher-student conversations in the classroom, and then further analyze the teacher's teaching attitudes and teaching styles based on the code categories. 11 Classroom Dialogue Classification Frequency Statistics Table (Total Observation Period: 1200″) Teaching with student-led orientation ⑨ Item Frequency 2 times Percentage 0.17 2.67 1.75 1.0 5.58 Ratio 36.42 16.75 7.17 0.33 33.75 58 32 21 12 67 72 365 201 86 4 405 696 ① ① ② Total ⑩ ④ ⑤ ⑥ ⑦ ⑧ Count Neutral Teaching that reflects teacher autonomy Combined The above table shows that the teacher-led approach in the classroom is absolutely dominant (categories 5, 6, 7, and 8, 58%), and that teacher-led teaching is less likely to deviate from the theme of the lesson, and encourages the students to respond in a timely manner to the teacher's expectations (category 8). Students' autonomy-oriented teaching is less frequently used (category ①, ②, ③, accounting for only 5.58%), which is not conducive to the cultivation of practical ability and innovative consciousness. It can also be seen that the teachers have clearly demonstrated a certain degree of teaching democracy (category 7, 4.33%), and they have paid attention to the use of encouraging evaluation (category 2, 1%). As a matter of fact, the teacher-led teaching method is absolutely dominant in the current teaching. The reason for this is that, in addition to the objective influence of the teaching content, it is mainly influenced by the traditional teaching in our country for thousands of years, and it is difficult to change the traditional teaching concepts. 12 3. Using the TIMSS Classroom Video Analysis Method Classroom observation using the TIMSS classroom video analysis method is a modern method. The Modern Education Laboratory of the Shanghai Institute of Educational Sciences gives the following steps for classroom observation and analysis with the help of classroom video: (1) Establish a coding system for classroom video analysis (2) Convert the video into a digital file that can be easily processed by the analyzing software (3) Code the various features of classroom teaching according to the coding system (4) Statistically analyze the results of the coding process (5) Conduct a secondary analysis on the basis of the results of the preliminary analysis (6) Generate classroom observations of the classroom teaching and learning. (5) Secondary analysis based on the preliminary results (6) Generation of classroom teaching analysis report Mr. Miao Rong from Peking University also analyzed the video lessons of two teachers in Jinan using the TIMSS method. The classroom is a complicated laboratory, and classroom observation is also a complicated project. The vision and means of observation should be open and diversified, and the content of observation should cause teachers and researchers to think y about the problems in teaching. Let's go into the classroom, observe with our heart, and gradually make classroom observation, classroom reflection and teaching practice become unified. 1Gary D. Borich, translated by Gary Mou and Zhang Zeli. The Cultivation of Teachers' Observational Skills - The Way to Highly Effective Teaching. Beijing, China Light Industry Press, 2006. 2 Thomas L. Good, Jere E. Brophy, Tao Zhiqiong, Wang Feng, Deng Xiaofang. Classroom in Perspective, Beijing, China Light Industry Press, 2002. 13 3 Contemporary Educational Science, by Cui Yun Kuo, Shen Yi, etc. 4 Classroom Observation Guidelines, by Chen Yun Kuo. 4 Guide to Classroom Observation, by Chen Yao, Educational Science Press, 2002 edition. 5 Teaching Theory by Shi Liangfang, East China Normal University Publishing House, 1999. Attachment: Evaluation Report of the Reflection Group on Self-directed Study Group Members: Ji Shiwei 2007101344 Zheng Weixin 2007101346 Zheng Bixia 2007101308 Zheng Peisi 2007101311 Zhong Mingjun 2007101302 This semester under the guidance of Ms. Li Jing, we attempted to conduct a very special study on this topic. In the initial stage of topic selection, we encountered some challenges. We searched and searched for a new topic that would allow us to complete our homework quickly and well. We finally chose "How to Conduct Effective Classroom Observations" because it was at least somewhat new to us and the difficulty factor should be moderate, and we began our research on the topic. So we started our research. We tried to go to the library to look for information, but the result was a bit disappointing, after all, we did not find a lot of information, we could only go to the Internet to look for journals or other types of information, finally, with the help of the group to basically complete the task of the preparatory stage, the encounter also made us realize that if we want to look for information, we must find it at the first time, and we can be better prepared if we actively face the topic. As the study progressed, we realized that our initial desire to complete the project quickly and well had changed into a desire to understand each part of the project carefully, and we gradually fell in love with the project.14 The members of our group combined the information we had found with the information we had found to make the project a success. 14 Our team members worked together to complete the first draft of the paper, taking into account the information we had found and our understanding of the topic. When the first draft was finished, it had 6,000 to 7,000 words, which is not a problem for a thesis, but considering the teacher's requirements and the constraints on the production of the courseware and the time for lectures in the later stage, we reduced the first draft considerably, and in the process of reducing the draft, we were faced with the main problem of: how to make sure that the revised thesis can highlight the key points and take into account the overall situation under the constraints of the word count? In the first draft, we analyzed how to carry out effective classroom observation in four parts: the formulation of the problem, the elaboration of the concept, the introduction of the method, and the reflections and suggestions. During the first revision, we mainly deleted the paragraphs with too much discussion in each section, for example, about 700 words were deleted from the introduction about the development history of classroom observation. After the first revision, we still could not meet the requirements, so we listened to the teacher's suggestions and made a second revision, deleting the fourth part. Although the revised paper was still structurally complete and logically rigorous, it was worthwhile to reflect on the fact that we did not take the word limit into consideration in the early stage of writing, and there were still shortcomings in the deletion of materials that could not be fully emphasized. After we finished the group paper, we entered another stage, that is, the production of the courseware and the preparation of the lecture. At the beginning, we thought that this stage would be easier, but it was not what we thought it would be. In the production of the courseware, we adopted a different production method from the other students, changing the way we used to use PPT and replacing it with an e-magazine. The idea was good, but it was very hard to do it. The students stayed up for several nights to make this courseware. Every time we finished making it, we found some problems. For example, some of the text didn't fit on the same page; some of the boxes were too small and the fonts were too big, so some words were missing? There were so many problems that we had to revise it again and again until all the members of the group were satisfied. Thinking about it now, it was really hard for the students who worked on the e-zine at the beginning. As for lectures, we originally thought that as long as we had done a good job with the courseware, it would not be a problem to lecture with the courseware. However, our idea was proven wrong by other students in the first class. It was only in the second class that we delivered the lesson, which brought us a lot of advantages, and the evaluation of other students in the first class as well as the teacher's suggestions provided us with valuable advice. When we came back, we immediately thought that only when we knew the content of the lecture by heart could we deliver it well and give our own insights. After we had prepared the lesson and familiarized ourselves with the content of "How to Conduct Effective Classroom Observation", we were once again faced with another problem, that is, how to make the lesson clear and understandable to the students. For this, we had countless discussions and tried to deliver the lesson many times. However, the result was still unsatisfactory. When we listened carefully to the members of our group, we found some problems that we hadn't noticed before. First of all, the speaker's voice, which we had not considered before and for which we had not made good preparations; secondly, we found that the courseware was beautiful but too beautiful, which distracted the audience from a large part of the courseware, and the speaker's voice was not loud enough, which made the audience rely on the courseware while trying to listen to the lecture; and thirdly, the production of the courseware did not highlight the content of the lecture, that is to say, the content of the lecture, which was not well presented. Again, the courseware was not created in such a way as to emphasize the content of the lecture, which meant that the focus was not obvious and prominent enough; and finally, the subject matter was not grasped very well. The topic we talked about was a very broad one, and it was also a topic with a lot of theories, and we felt that we were not able to grasp the main points of the topic. On the whole, we felt that our group had done a good job in preparation. Although the speaker's voice was small, her familiarity with the classroom materials made her speak in an organized manner, and she was able to bring in the appropriate movements and expressions, which would have been better if we had not been unable to use the loudspeaker temporarily, and if we had thought about this before. After this research group cooperation, we have gained a lot, it not only enhances our cooperation ability and research ability, but also makes us have a new understanding of educational evaluation and measurement, which is different from the theory, we will learn from the lessons of this research, so that other research activities in the future will be more perfect and outstanding,