Development of a Competence Model for Object-Oriented Programming
Project Description
Cooperation with Informatics Education Research, Munich
Affected by the national results of PISA, in the past years lots of educational fields established the modelling of competences as one of their research focusses. The German Research Foundation (DFG) reacted by implementing the Priority Program 1293 "Competence Models for Assessing Individual Learning Outcomes and Evaluating Educational Processes", in which several topics from different educational fields were subsumed. In the last 7 years of Informatics Education research two projects addressed this topic. On the one hand the DFG-project MoKoM researched, how competences for Informatics Systems and Modeling could be described. On the other hand, KUI (granted by the federal ministry of education and research (BMBF)) analyzed, which competences future educators in the area of Informatics should possess. For further fields of content of empirically founded models are not known yet.
One possible field of content, which is quite relevant in german grammar school curricula as well as in introductory courses at universities, is the introduction to object-oriented programming (OOP). In schools, students learn via programming to understand, enhance or independently design informatic systems, which were beforehand analyzed and built for problem statements derived from their everyday lives. This also aims at evaluating the possibilities and limitations of these systems as well as systematically enhance their problem solving skills. Because of the practical relevance the introduction to programming using an object-oriented language is a very common approach at universities, too.
To understand the processes of teaching and learning in this field better and to make them accessible for a comparison on a larger scale, it is the aim of this project to develop a competence model for the secondary school field along with its assigned measuring instruments for a relevant part of the area of object-oriented programming. The acquisition of programming skills confronts beginners often with major problems, since the knowledge of programming language syntax and semantics, of handling the tools (e.g. integrated development environment), of programming plans and of problem solving have to be interconnected continuously. Such a competence model could also contribute in enhancing the processes of teaching and learning as well as the accompanying teaching materials.
Since latent structures such as competences reveal themselves e.g. through the solving of certain problems, typical tasks in the field of introductory programming in teaching books, hand-outs and introductory lectures will be analyzed and categorized regarding the required abilities to solve the task in a competent way. Based on this categorization a normative competence structure model will be derived. The sufficient completeness and correctness of this model will be confirmed by an evaluation of experts in the field of pedagogical content. By collaborating with representatives of psychology and psychometrics, the normative model will be transferred into a psychometric measuring model. This model will be the basis for upcoming items, which will be formulated, piloted, enhanced and finally tested on a representative cohort. The resulting findings will be enriched with the drawing of concept maps. The evaluation of the item quality features as well as the distinguishing of item difficulty and person ability will be done with the methods of Item Response Theory. The results will cover the previous findings on an empirical level.
- Brinda, T.; Kramer, M.: Competency models in computing education. In: Tatnall, A. (Ed.): Encyclopedia of Education and Information Technologies. Springer, Cham 2019, p. 330-333. doi:10.1007/978-3-319-60013-0_5-1CitationDetails
- Striewe, M.; Kramer, M.: Empirische Untersuchungen von Lückentext-Items zur Beherrschung der Syntax einer Programmiersprache. In: Bergner, N.; Röpke, R.; Schroeder, U.; Krömker, D. (Ed.): Hochschuldidaktik der Informatik - HDI 2018. Universitätsverlag Potsdam, Potsdam 2018, p. 101-115. Full textCitationDetails
- Kramer, M.; Barkmin, M.; Brinda, T.: Evaluating Responses in Source Code Highlighting Tasks - Preliminary Considerations for Automatic Assessment. In: Mühling, A.; Cutts, Q.; Schwill, A. (Ed.): Proceedings of the 13th Workshop in Primary and Secondary Computing Education (WIPSCE 2018), Potsdam, Germany, 4.-6. Oktober 2018. ACM Press, New York 2018, p. 1-4. doi:10.1145/3265757.3265775Full textCitationAbstractDetails
The reading and understanding of source code is a necessary prerequisite for prospective programmers to write their own code. Thus, a reasonable task is to determine certain concepts like class identifiers in a given source code. In order to automatically evaluate the responses, they must be quantified. Hence, it is vital to have reliable and valid methods of measurement. In this paper we apply different means of measurement on constructed examples and compare them. Cohen's κ seems to be an adequate measure.
- Kramer, M.; Barkmin, M.; Tobinski, D.; Brinda, T.: Understanding the Differences Between Novice and Expert Programmers in Memorizing Source Code. In: Tatnall, A.; Webb, M. (Ed.): Tomorrow's Learning: Involving Everyone. Learning with and about Technologies and Computing 11th IFIP TC 3 World Conference on Computers in Education, WCCE 2017, Dublin, Ireland, July 3-6, 2017, Revised Selected Papers. Springer, Cham, Switzerland 2018, p. 630-639. doi:10.1007/978-3-319-74310-3_63Full textCitationAbstractDetails
This study investigates the difference between novice and expert programmers in memorizing source code. The categorization was based on a questionnaire, which measured the self-estimated programming experience. An instrument for assessing the ability to memorize source code was developed. Also well-known cognitive tests for measuring working memory capacity and attention were used, based on the work of Kellog and Hayes. Thirty-eight participants transcribed items which were hidden initially but could be revealed by the participants at will. We recorded all keystrokes, counted the lookups and measured the lookup time. The results suggest that experts could memorize more source code at once, because they used fewer lookups and less lookup time. By investigating the items in more detail we found that it is possible that experts memorize short source codes in semantic entities, whereas novice programmers memorize them line by line. Because our experts were significantly better in the performed memory capacity tests, our findings must be viewed with caution. Therefore, there is a definite need to investigate the correlation between working memory and self-estimated programming experience.
- Barkmin, M.; Tobinski, D.; Kramer, M.; Brinda, T.: Code structure difficulty in OOP: an exploration study regarding basic cognitive processes. In: Proceedings of the 17th Koli Calling Conference on Computing Education Research . ACM Press, New York 2017, p. 185-186. doi:10.1145/3141880.3141913Full textCitationDetails
- Barkmin, Mike; Kramer, Matthias; Tobinski, David; Brinda, Torsten: Unterschiede beim Memorieren von Quelltexten zwischen NovizInnen und ExpertInnen der objektorientierten Programmierung. In: Diethelm, I. (Ed.): Informatische Bildung zum Verstehen und Gestalten der digitalen Welt (GI-Fachtagung "Informatik und Schule - INFOS 2017", 13.-15.09.2017 in Oldenburg). Köllen, Bonn 2017, p. 407-408. Full textCitationAbstractDetails
Dieser Artikel befasst sich mit den unterschiedlichen Vorgehensweisen von ExpertInnen und NovizInnen der Programmierung beim Memorieren und Interpretieren von Quelltexten. ExpertInnen haben die Quelltexte überwiegend auf einer semantischen Ebenen memoriert, NovizInnen hingegen zeilenweise.
- Kramer, M.; Samimi, V.; Brinda, T.: Entwicklung eines Online-Tools zur Bestimmung objektorientierter Programmierkompetenzen. In: Diethelm, I. (Ed.): Informatische Bildung zum Verstehen und Gestalten der digitalen Welt (GI-Fachtagung "Informatik und Schule - INFOS 2017", 13.-15.09.2017 in Oldenburg). Köllen, Bonn 2017, p. 425-426. Full textCitationAbstractDetails
Basierend auf einem theoretisch hergeleiteten Kompetenzmodell für objektorientierte Programmierung wird ein Prototyp eines Messwerkzeuges zur empirischen Bestimmung solcher Kompetenzen vorgestellt. Erste Tests existieren bereits, zukünftige Tests werden skizziert.
- Kramer, M.; Tobinski, D.; Brinda, T.: Modelling Competency in the Field of OOP: From Investigating Computer Science Curricula to Developing Test Items. In: Processing, International Federation For Information (Ed.): Stakeholders and Information Technology in Education. Springer, Cham, Switzerland 2016, p. 37-46. CitationDetails
- Kramer, M.; Tobinski, D.; Brinda, T.: On the Way to a Test Instrument for Object-Oriented Programming Competencies. In: Proceedings of the 16th Koli Calling International Conference on Computing Education Research. ACM Press, New York 2016, p. 145-149. doi:10.1145/2999541.2999544Full textCitationDetails
- Kramer, M.; Hubwieser, P.; Brinda, T.: A Competency Structure Model of Object-Oriented Programming. In: 2016 International Conference on Learning and Teaching in Computing and Engineering (LaTICE). IEEE, Mumbai 2016, p. 1-8. Full textCitationDetails
- Brinda, T.; Hubwieser, P.; Kramer, M.; Ruf, A.: Towards a Competency Model for Object-Oriented Programming. In: Dagiene, V; Schulte, C.; Jevsikova, T. (Ed.): Innovation and Technology in Computer Science Education - ITiCSE - 2015. ACM New York, NY, USA, 2015, p. 345-345. doi:10.1145/2729094.2754848Full textCitationDetails
- Kiesmüller, Ulrich: Prozessbegleitende automatisierte Identifizierung von Problemlösungsstrategien Lernender beim Lösen algorithmischer Probleme mit Programmierumgebungen in der Sekundarstufe I. Shaker Verlag, Aachen 2013. CitationDetails
- Brinda, T.; Puhlmann, H.; Schulte, C.: Bridging ICT and CS - Educational Standards for Computer Science in Lower Secondary Education. In: Acm (Ed.): Proceedings of the 2009 ACM SIGCSE Annual Conference on Innovation and Technology in Computer Science Education (14th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE 2009), Paris). ACM Press, New York 2009, p. 288-292. CitationAbstractDetails
Recently, the importance of computer science education in secondary schools has been coming more and more into focus. Students' interests and motivation to pursue a career in CS related fields are highly influenced by school. Also beliefs in the nature of CS are influenced, and necessary foundations in knowledge and skills are built. However, a major problem is the gap between information and communication technologies (ICT) and computer science (CS). Often pupils have only experiences in ICT, and therefore develop inadequate beliefs about CS. We propose educational standards for CS in lower high school in order to bridge ICT and CS.
- Schulte, C.; Brinda, T.: Beiträge der Objektorientierung zu einem Kompetenzmodell des informatischen Modellierens. In: Friedrich, S. (Ed.): Unterrichtskonzepte für informatische Bildung. Köllen, Bonn 2005, p. 137-148. CitationAbstractDetails
In zwei informatikdidaktischen Forschungsprojekten (Laufzeit 2000-2004) in NRW wurden Konzepte zum objektorientierten Modellieren im Informatikunterricht der Sekundarstufe II entwickelt und erprobt. Dabei wurden Erkenntnisse über Erfolge und Schwierigkeiten von Lernenden in diesem Bereich gewonnen. Auf der Grundlage dieser Ergebnisse werden im vorliegenden Beitrag mit dem Ziel der Entwicklung eines Kompetenzmodells Schlussfolgerungen zur Identifikation und zur Beschreibung von Kompetenzen und möglichen Abstufungen in diesem Bereich abgeleitet.
- Brinda, T.: Preparing educational standards in the field of object-oriented modelling. In: Magenheim, J.; Schubert, S. (Ed.): Informatics and student assessment. Concepts of empirical research and standardization of measurement in the area of didactics of informatics. Köllen, Bonn 2004, p. 11-22. CitationDetails
- Brinda, T.: Integration of new exercise classes into the informatics education in the field of object-oriented modelling. In: Education and Information Technolgies, Vol 9 (2004) No 2, p. 117-130. CitationDetails
- Brinda, T.: Didaktisches System für objektorientiertes Modellieren im Informatikunterricht der Sek. II (Dissertation). Fachbereich Elektrotechnik und Informatik, Universität Siegen 2004. CitationDetails
Vorträge
- Barkmin, M.; Kramer, M.: Automatische Auswertung von Leistungserhebungen zur unmittelbaren Diagnostik im (Informatik-)Unterricht; Kolloquium des Lehrstuhls Metheval, 15.01.2020, Jena. Details
- Kramer, Matthias: Nachhaltige informatische Bildung für die Datengesellschaft - Eingeladener Teilnehmer der Podiumsdiskussion; 2. PIN-Lab Symposium 2018, 30.11.2018, Universität Paderborn. Details
- Kramer, Matthias: Kompetenzmodellierung im Bereich der objektorientierten Programmierung; 65. Bremerhavener MNU-Tagung, 20.11.2018, Bremerhaven. Details
- Barkmin, Mike; Kramer, Matthias: Automatische Auswertung von Leistungserhebungen zur unmittelbaren Diagnostik im (Informatik-)Unterricht; MINT-Kongress, 08.09.2018, Universität Duisburg-Essen. AbstractDetails
In dem Workshop soll anhand eines webbasierten Tools, welches vom Lehrstuhl für Didaktik der Informatik entwickelt wurde, aufgezeigt werden, wie komplexere Aufgabentypen durch automatische Auswertung in den Unterricht zur unmittelbaren Diagnostik genutzt werden können. Die Teilnehmenden werden zunächst die verschiedenen Aufgabentypen kennenlernen und im späteren Verlauf die automatisch ausgewerteten Abgaben analysieren und zur Diagnose unter verschiedenen Gesichtspunkten einsetzen. Abschließend soll mit den Teilnehmenden diskutiert werden, inwiefern sich die vorhandenen Aufgabentypen und automatischen Analysen für den (Informatik-)Unterricht eignen und welche zusätzlichen Analysen und Aufgabentypen aus ihrer Sicht für eine gute Diagnose sinnvoll sind.
- Barkmin, Mike; Kramer, Matthias: Entwicklung eines Online-Tools zur Bestimmung von Programmierkompetenzen - Aktueller Zwischenstand; Oberseminar "Didaktik der Informatik", 24.01.2018, Essen. Details
- Barkmin, Mike; Kramer, Matthias: Towards Automatic Competency Assessment of Programming Skills - Preliminary results of a current development; IZfB "Scientist in Residence" Symposium: "Digitalisation and the Future of Higher Education and Work", 05.12.2017, Essen (Posterbeitrag). AbstractDetails
Background
Due to the results of international student assessment studies like PISA, the normative determination and empirical verification of competencies are topics in the focus of educational and psychological research during the last years. In the area of object-oriented programming the project COMMOOP focuses on the identification of such competencies. Currently, in the framework of this project a test instrument for comparative studies of larger cohorts as well as for individual diagnostics is under construction.
Following a comparing analysis of various definitions of the term *competency*, we understand competencies in the area of object-oriented programming as domain specific cognitive and metacognitive abilities and skills that enable individuals to solve problems in the area of object-oriented programming. During an intensive literature research a four-dimensional competency model could be derived. First steps towards verification were done by interviewing experts and integrating competency formulations from national and international curricula. Furthermore 12 test items were derived and piloted in a first pen and paper test. This format has been shown resulted in a quite inefficient and time-consuming procedure due to the process of digitalization and evaluation as well as due to the reach-out for further test subjects. It was thus considered detrimental. Hence, it was reasonable to seek an alternative approach in a digital solution.
Implementation & current status
Considering the situation that teachers often do not have administrative rights and furthermore to avoid installation issues we decided to develop a web-based application, which could be run in any web browser on any device with internet access. The web application is independent from the field of computer science and could be used in other fields as well. So there are simple generic item formats, like multiple choice and questionnaire. But there are also special item formats, which were custom-built for accessing programming competencies. Because some item formats are more complex than others requiring more than one step to complete, it would not been enough to use the final submission for enhancing the items. So we implemented a generic way to record all actions the user has taken during the process. These recordings could be helpful to explore cognitive processes while solving the tasks. Given an inside into difficulties, the improvement of the item pool will be facilitated. To make it easy to create new items and tests, we have implemented a graphical user interface for this purpose.
Potential and outlook
The tool has the potential to let interested researchers assess and evaluate the competency level of students with regard to a competency model. Especially the recording of all actions could be very helpful to get an insight into how students deal with the item, to know where they struggle. In the future, when we have collected enough data, we will implement adaptive tests to assess the competency of our pupil with a far better accuracy.
- Kramer, Matthias: Entwicklung eines Kompetenzstrukturmodells für objektorientierte Programmierung in der Sekundarstufe II - Schwierigkeiten bei der Konstruktion von Items und die Notwendigkeit einer automatisierten Testumgebung; Doktorandenkolloquium Informatik 2017, 16.03.2017, München. Details
- Kramer, Matthias: Entwicklung eines Kompetenzstrukturmodells für objektorientierte Programmierung in der Sekundarstufe II - Entwurf einer Kompetenzstruktur und Präsentation darauf basierender Items; Doktorandenkolloquium Informatik 2016, 29.02.2016, Koblenz. Details
- Kramer, Matthias: Entwicklung eines Kompetenzstrukturmodells für objektorientierte Programmierung in der Sekundarstufe II; Fachleitertagung Informatik 2015, 20.10.2015, Essen. Details
- Kramer, Matthias: Entwicklung eines Kompetenzstrukturmodells für objektorientierte Programmierung in der Sekundarstufe II - Ergebnisse qualitativer Untersuchung von Schulbuchaufgaben und weitere Schritte; Doktorandenkolloquium Informatik 2015, 25.03.2015, Aachen. Details
- Kramer, Matthias: Entwicklung eines theoretisch fundierten und empirisch abgesicherten Kompetenzmodells zur objektorientierten Programmierung - Erste Ansäzte mittels qualitativer Auswertung von Lehrbuchaufgaben; Doktorandenkolloquium Informatik 2014, 02.10.2014, Paderborn. Details
- Barkmin, Mike: Entwicklung eines Messinstruments zur Erfassung von Kompetenzen in den Dimensionen OOP Wissen & Fähigkeiten und Umgang mit Repräsentationen - Teilvaliderung eines Kompetenzstrukturmodells zur OOP; Doktorandenkolloquium Informatikdidaktik 2018, 03.10.2018, Potsdam. Details