A Questionnaire-based survey

Available under Creative Commons-ShareAlike 4.0 International License.

The questionnaire-based survey focuses on trainers’ and trainees’ opinions about the utility of the system. The questionnaire contained 10 questions, graded using a 5 points - Likert scale (from “I strongly disagree” to “I strongly agree”). 16 users filled up the questionnaire: 5 tutors (who created rules-based tests) and 11 students. The questions are available in Table 8.2. When designing the questionnaire, the directives of the Australian Flexible Learning Framework were taken into account: they drew the attention to the the tendency of developers and suppliers of e-assessment applications to be concerned only with the technical aspects of the products, neglecting information about the success and usefulness of products in the learning process per se (Australian Flexible Learning Framework, 2011).

All participants agreed that the interface is easy to use, because it is intuitive and it applies the same conventions in all screens. The application is considered quite useful, since it fulfills the established educational objectives and verifies the knowledge gained from assimilation of educational materials provided by the International Project Management Association. Most trainers concluded that the application needs more features: existence of predefined tests, more support to create tests, the possibility to add short videos for the students to be able to react to certain work conduits. Most of the participants agreed that the application was stable; the only problem was the duration of the .NET session, which was too short. This shortcoming was solved by increasing the session duration on the server. The security of the application didn’t raise any problems. Most participants consider the feedback to be the most useful feature of the application: 7 out of 11 students strongly agreed on the importance of feedback, 2 agreed and 2 were indifferent. Just 2 out of 11 acknowledge the utility of the adaptive tests: students don’t realize that the feedback mechanism is well performing thanks to the adaptive algorithms. This finding can be explained by the fact that technical details are hidden to the students. Most trainers are satisfied with the rules creation interfaces. Just one of them said that he would like to build criteria for rules creation.