Examination Design

The added value of written exams is relatively clear: the framework is the same for everyone, as are the questions and the grading system. In addition, it is possible to obtain proof of performance from many students in a short period of time.

Written examination formats are much more diverse than is commonly thought: from the typical written exam with open and closed or multiple-choice questions to written seminar papers and learning portfolios in which students are supposed to reflect on their own learning process. Within the various examination formats, there are many different question formats - from pure knowledge questions to application-oriented tasks – which, based on the learning objectives taxonomy, enable precise testing of the desired learning outcomes.

When selecting an examination format, it is always important to first compare it with the examination regulations. These specify which examination formats are permissible and how their use is regulated.

In order to make a written examination as fair and objective as possible, the requirements for the examinees and the framework conditions should be made transparent at an early stage. Old exams or sample tasks can be provided for this purpose. One danger of written examinations is that only a comparatively one-sided picture of the examinee is obtained, only a small section of what has been learned can be tested and it is difficult to deal with the particularities of the examinee (test anxiety or similar issues). To ensure objectivity, you as the examiner must define clear assessment criteria in advance and be clear about which answers you expect as a sample solution. You should also consider whether and how the questions are interdependent and how you will deal with consequential errors.

Tips:

Multiple choice tests allow exams with a large number of participants to be held and evaluated quickly and reliably. Suitable software supports not only the creation of the questions but also the evaluation of the exam, feedback to the students and the checking of the task quality. The quality of MC questions can be checked well by means of statistical evaluations that can be automated and can thus also be secured and improved more easily; test quality criteria such as objectivity and reliability are generally high. One can also easily measure difficulty and validity. And leaving aside the fact that not everyone likes every form of testing, this type of test is relatively fair.

The main disadvantage of this format is that the positive features of this type of test can only be realized with carefully formulated questions and answer alternatives ("distractors"). Otherwise, there is a risk of arriving at the correct solutions by guessing or combining. Moreover, MC questions are not suitable for all levels of learning outcomes. Whereby MC tests can definitely ask for more than just memorized knowledge! There are enough possibilities to ask not only simple true-false questions and knowledge queries but also assignment questions, questions about preconditions and consequences, analogies or questions as small case studies.

Plausible and well-worded distractors, along with interesting questions, are the essence of good MC exams. This way, you may not solve the guessing problem, but you eliminate the hidden clues due to weaknesses in content or language. No MC question should be crackable because you understand the principle of its construction.

Legal problems and implementation guidelines derived from them are always a big topic. In the General Academic and Examination Regulations (APSO) (§ 12a) you can find detailed guidelines for MC exams. For TUM in general the following applies: There may be – with at least three, but recommended four or more answer options – only one correct answer (single choice). There are 0 points for wrong answers, no points ("malus points") may be deducted.

Tips:

• Create a topic grid. List the content to be tested in full and then specify the desired processing depth of the tasks in the columns. In addition, you can specify the weighting of the individual topics.
• Begin formulating the answer alternatives with the correct answer and then continue with the incorrect ones. Arrange the answer alternatives logically, according to solution content in ascending or descending order, alphabetically ...
• Formulate simply, clearly and positively; do not forget the information necessary to answer! Answers should be grammatically appropriate to the question and should not differ in length or differentiation.
• Reduce the likelihood of pure guessing by using a large enough number of questions with as many (4 or more) alternative answers as possible.
• Always pretest with the questions and evaluate for goodness of fit. Also test the questions with non-specialists for comprehensibility, hidden clues, etc.
• Create a pool of questions at the chair and add to it regularly. Check the questions occasionally or evaluate the test results according to question quality. Shuffle the test questions so that they do not appear in the same order.

Oral exams cover a wide spectrum: from the typical individual oral exam to reports or oral presentations to group exams. Regardless of the format used for the oral examination, proof of performance in the form of written documentation such as a protocol is essential. In the case of group examinations, it is also important that the performance of the individual is recognizable and reflected in the individual grade. In addition, when selecting the examination format, it is essential to compare it with the requirements of the examination regulations.

In order to make the examination as fair as possible and to reduce the "examination anxiety" factor, the requirements for the examinee and the general conditions should be made transparent at an early stage. One way of doing this is to provide sample questions or to publicize criteria for determining grades.

In order to make the assessment of examinations as objective as possible, it is important that you as the examiner define clear assessment criteria in advance and that you are clear about which answers you expect as sample solutions and to what extent. Especially in the case of oral examinations such as papers and presentations, the student should be informed about which criteria, in addition to the correct presentation of the content, are included in the evaluation, e.g. eloquence, form of presentation, scope of the explanations ... In addition, it is helpful to reflect on one's own expectations before and during the examination and to pay attention to which criteria actually reflect the student's performance and which rather reflect personal preferences of the examiner.

The added value of oral examinations compared to written examinations lies on the one hand in the possibility of obtaining a comprehensive picture of the examinee. On the other hand, special features of the situation or the examinee (such as test anxiety or stage fright) can be dealt with flexibly.

Tips:

If you think about competency orientation in a consistent way, you quickly end up with practical examinations. Above all, students can prove that they have not only acquired inert knowledge, but are also capable of translating knowledge, skills and attitudes into competent action. Initial experience with these competence-oriented examinations shows that they are very time-consuming – but that they allow reliable statements to be made about the competencies acquired, effectively control the learning process, are instructive, and are often even fun - for both students and instructors.

In order to test competencies, students must be given the opportunity to appropriately deal with a situation that is as close to real life as possible using their knowledge and skills. Creating such situations that are as realistic as possible and translating them into an examination format is unfamiliar and time-consuming. Elements of a practical exam can be:

• Role plays
• Simulations, business games
• Building models, programming
• Circuit stations where skills are demonstrated

In medicine, for example, there is the "Objective Structured Clinical Examination" (OSCE) format. It consists of a course with various standardized stations where students demonstrate practical skills (e.g., applying a plaster cast, placing a catheter, conducting an educational interview) on simulated patients, fellow students, or simulation mannequins and are observed by the examiners and graded on the basis of simple rating scales. Such an exam can take a full day for 100 students and employ around 20 examiners and assistants.

Tips:

If students wish to have externally acquired achievements recognized, the respective subject representatives must assess whether the learning outcomes are equivalent and the achievement can be recognized at TUM. But what does equivalence mean? What are learning outcomes and how can they be identified and compared?

The legal framework for the recognition of study and examination achievements and the recognition process at TUM are formally regulated. However, open questions always arise when subject representatives (usually module supervisors) have to assess the equivalence of achievements.

The TUM Center for Study and Teaching has therefore developed a handout to support all those involved in the recognition process in assessing the equivalence of study and examination achievements.

The handout

•  specifies the legal framework and terms related to recognition,
• provides tips and examples on how to check the equivalence of learning outcomes in terms of content and level in practice,
• explains how credits and grades from the external institution are handled,
• summarizes what has to be considered when a request for recognition is rejected,
• gives an overview of which organizational aspects have to be considered when applying for recognition,
• shows how a Learning Agreement promotes the mobility of TUM students.

You can download the handout in mytum.

Many instructors are unsure how to correctly deal with the review of graded exams. The TUM Center for Study and Teaching has summarized the legal requirements in the publication "Recommendation on Review of Graded Exams" and provides recommendations on how to plan and conduct reviews of graded exams in practice.

You can download the document in mytum.

How often can exams be repeated? And when do repeat exams have to be offered?

You can find an overview of the legal framework for conducting repeat examinations in mytum (German).

In order to ensure a sensible and efficient allocation of lecture halls for courses, examinations and conferences, we have summarized all important rules for booking here. Please also note the allocation principles (PDF, 2.5 MB) (German) and the key data for the allocation of lecture halls and seminar rooms (German).

Room allocation during the semester

As employees of the departments and schools, you can have lecture halls that are centrally managed reserved for lectures and seminars during the semester. You can find out about lecture hall reservations in advance in TUMonline.

Room allocation during the lecture-free period

During the lecture-free period, seminar rooms and lecture halls can (also) be assigned to (external) interested parties for colloquia, seminars, examinations.

Please note that courses and exams always have priority over all other events. According to the EHL decision of 12.11.2013 (German) the following weeks are kept free for exams:

• the first three weeks after the end of lectures
• the two weeks before the start of lectures in the summer semester
• two of the three weeks before the start of lectures of the winter semester (depending on the Oktoberfest).

Nevertheless, conferences of “extraordinary importance” are conceivable at TUM in consideration of the implementation of preliminary courses in the defined conference windows. TUM defines periods for lectures, examinations, and conferences (German) at least three years in advance. Conferences can also be held in the defined time windows only if they are requested by e-mail from the responsible room allocation office at least one year before the planned start of the conference and if they are conferences of “extraordinary importance”.

Lecutre Halls in Munich and Garching

Internal:
Frank Baumann
Tel.: +49 89 289 22243
E-Mail: frank.baumannspam prevention@tum.de

Examination Planning Garching:
Vanessa Barcsa
Tel.: +49 89 289 28847
E-Mail: barcsaspam prevention@zv.tum.de

Samantha Steinberger
Tel.: +49 89 289 28848
E-Mail: samantha.steinbergerspam prevention@tum.de

External:
Monika Wagner
Tel.: +49 89 289 25307
E-Mail: wagnermspam prevention@zv.tum.de

Lecture Halls at the Campus im Olympiapark CiO:

Internal:
Birgit Birkeneder
Tel.: +49 89 289 24627
E-Mail: birgit.birkeneder@tum.de

External:
Renate Werschel
Tel.: +49 89 289 24701
E-Mail: werschelspam prevention@zv.tum.de
 

Lecture Halls in Weihenstephan:

Internal:
Gundula Striek 
Tel.: +49 8161 71 3364 
E-Mail: Gundula.Striekspam prevention@mytum.de 

External:
Marianne Röhrl
Tel.: +49 8161 71 3212
E-Mail: roehrlspam prevention@zv.tum.de