E-Assessment System for C/C++ Programming Assignments
Motivation and Need
In any educational system, no one can deny the importance of assessments. Assessments help in evaluating the knowledge gained by a learner at any specific point as well as in continuous improvement of the curriculum design and the whole learning process. There are broadly two types of assessments, i.e., formative and summative. The summative assessment gauges the skills and knowledge gained by a student after the completion of some learning process (Ridgway et al., 2004). In formative assessment, feedbacks are provided to students for improving their abilities in any given domain (Ridgway et al., 2004).
In Pakistani higher education institutions, especially that follow semester system; teachers typically prepare and give quizzes, assignments, projects, and conduct periodic exams to achieve both formative and summative assessments. However, due to increasing number of students enrolling in these institutions, teachers remain under burden during whole semesters. Besides assessments, teachers also perform other essential tasks such as: preparing course content, conducting lectures, providing feedbacks on students’ work, conducting meetings with students to resolve their problems related to course, and exams invigilation. Moreover, they also strive to conduct research for their future prospects and for the common good of the society. Having mentioned the range and nature of tasks performed by a teacher, it is highly probable that a teacher’s time and effort might be compromised in performing any or all of these activities. Unfortunately, this situation is observed to be widely prevalent in Pakistan.
From the students’ perspective, they require quality in lectures and learning materials. They also require fair grading in assessments. They also need comprehensive, immediate, and continuous feedbacks on their work from their respective teachers to improve their learned knowledge and skills. The non-availability of appropriate time and effort from a teacher in performing these tasks can seriously degrade students’ learning motivations and quality of learning.
In most of the advanced countries, teachers take the help of paid teaching assistants (TAs) to help them in conducting various tasks related to teaching, such as: preparing exams and assignments, their marking, and sometimes lecture delivery. However, in a developing country like Pakistan, there is a lack of skilled people and resources to hire such TAs. Moreover, the conflict of interest situations cannot be easily tracked among TAs and students.
In Pakistan, we need serious efforts in addressing these issues to improve quality of education in higher education institutions. There is an imminent need to come up with new ideas to automate various teaching tasks to support effective teaching and students’ learning. One task that can be automated is assignments checking and grading. Such automated assessments are usually called as Electronic Assessment (e-assessment) or Computer Aided Assessment (CAA). Some core examples of e-assessments strengths over existing systems are: “immediate feedback”, “adaptive testing”, “on-demand testing” (testing at any time and at any place) (Ridgway et al., 2004), quick marking and grading (time saving), reduced cost (TAs hiring), better collaborations between students, and uniformity in grading. Although, assignments for all courses cannot be checked automatically, but the courses that have some strict language of expression can be automated, for example, programming courses in the Computer Science domain.
Programming is an essential part of Computer Science and in even other academic disciplines. Practice is the only key to learn programming for which teachers provide assignments carrying practical problems to be solved. However, their evaluation requires significant time and effort. In distance/e-learning environment this situation is more prevalent due to large number of enrolled students and requires immediate attention. Assessment of these students requires large number of tutors. In such environment, quizzes and peer assessments are the only easiest options. In Pakistan, Virtual University of Pakistan (VUP) employs many tutors for such assessments costing it millions of rupees annually. The availability of automated tools to assist the checking of assignments can greatly reduce this cost.
In terms of time, we can explain the amount of time required for assessment with an illustration. For example, in COMSATS, each class has around 50 students. A teacher should give at least 3 assignments in a course. Hence, around 150 assignments need to be checked per course. Usually a teacher delivers three courses per semester. If for example, assignments consist of 25 problems (presumably a modest number) then total problems will be 25*150=3750. Suppose, if a teacher consumes 5 minutes on a single problem then approximately 18750 minutes (312.5 hours, ~40 working days per semester) will be required to check all assignments which is quite large and can be spent on other teaching aspects. The use of automated assessment tools can really help in saving teachers’ time.
The e-assessment system of VUP is the most “comprehensive” systems being used in Pakistan (Sangi & Malik, 2007). However, this system only checks MCQs and true/false questions automatically. We believe that it is time to move further for more advanced automated assessment systems to gain immediate advantages of e-assessments. The current proposed project is one of the efforts in this direction where we intend to automate the assessment of programming assignments.
This project aims at developing an automated assessment system for the introductory computer programming courses. In Pakistan usually C and C++ languages are being opted for teaching basic programming skills. Therefore, in this project we will focus on both C and C++ programming languages. The proposed product will primarily support formative assessments. The product will be based on different modules that are necessary for building an effective automated assessment system. These modules are: black box testing with automatic test data generation facility, static testing of source code, coding style analysis, grading on the basis of semantic similarity of students solutions with model solution, and source code plagiarism detection. In addition to this, we will build our product by using service oriented architecture to support loose coupling between modules/services, reusability of modules, and easy adaption in software in response to changes in business workflows. Moreover, this architecture will also help in making our product and its one or more above mentioned modules reusable for other applications. In this product, we will also use IMS Common Cartridge (IMS-CC, 2014) and IMS Digital Repositories Interoperability (IMS-DRI, 2014) standards that can help in standardized searching and sharing of assessment objects. Finally, we will implement a mechanism to support secure student-to-teacher and teacher-to-teacher collaboration.
We believe that our project will also provide valuable insights and hands on experience in tackling various issues practically (e.g., institutional “policies”, “procedures”, “assessment administration”, and “acceptance” (Sangi & Malik, 2007)) related to the deployment of the developed product in the context of Pakistani institutions, students, and teachers. This will ultimately pave the path of implementing such systems in future for other institutions in our home country as well as for the development and deployment of automated assessment systems for other courses such as: mathematics, algorithms analysis and design, databases etc.
In a nutshell, automated assessment systems are the need of the hour if we as a nation are committed in raising the standards of our educational system. In Pakistan, this project is a pioneer effort in this direction.
References
IMS-CC (2014). IMS Common Cartridge Specification. Retrieved from http://www.imsglobal.org/cc/, (Last Accessed: 14-Jul-2014)
IMS-DRI (2014). IMS Digital Repository Interoperability. Retrieved from http://www.imsglobal.org/digitalrepositories/driv1p0/imsdri_infov1p0.html, (Last Accessed: 14-Jul-2014)
Ridgway, J., McCusker, S., & Pead, D. (2004). Literature Review of E-assessment, FutureLab Series, ISBN: 0-9544695-8-5.
Sangi, N. A., & Malik, I. (2007). E-assessment models and methods for student evaluation, In Proceedings of 21st Asian Association of Open Universities Conference. Retrieved from asiapacific-odl2.oum.edu.my/C45/F473.doc