Armando Ordóñez¹, Martha Giraldo G.¹, Freddy Muñoz¹, Hugo Ordoñez², Yeni Rosero³

¹Intelligent Management Systems, University Foundation of Popayan, Popayan, Colombia.

Resumen

La personalización de la educación influye en la motivación de los estudiantes y mejora los resultados de las evaluaciones. Algunas herramientas informáticas han sido propuestas para automatizar la personalización tales como los tutores inteligentes de aprendizaje con excelentes resultados. Sin embargo, la mayoría de los trabajos existentes se centran en los estudiantes y dejan de lado a los docentes. En este trabajo, se presenta un framework de código abierto basado en un sistema de tutor inteligente. El framework busca reducir los costos de implantación y la complejidad de las interfaces. Igualmente, el framework considera la participación tanto de estudiantes como de docentes. El framework fue utilizado para construir un curso de matemática en primaria. El framework desarrollado servirá como base para modelar el aprendizaje en un curso SPOC.

Abstract

Personalized education contributes to the motivation of the students and improves student performance. Some tools such as the Intelligent Tutoring Systems have been proposed to this purpose with excellent results. However, most of the existing works have given little attention to the role of the teachers. In this article, an open source framework based on a standard intelligent tutoring system is presented. The framework aims at reducing the implementation costs and the complexity of the interfaces, in addition, the framework considers the participation of teachers. The framework was used to create a math course for an elementary school student, and will be used as a basis for the personalization of a Small Private Online Course.

1. Introduction

Learning management systems (LMS) offer significant advantages for the learning processes. However, the resources and the pedagogical strategies are the same for all students, leaving aside the personalization (Wentzel & Brophy, 2014).For this reason, the personalization in learning has attracted the attention of student and teachers for several years. Diverse studies have shown that personalized learning contributes to student motivation, and significantly improves student performance (Hwang, Sung, Hung, Huang, & Tsai, 2012). The main objective of the research community is to apply diverse technologies to achieve personalized instruction and better student learning outcomes (Walkington & Hayata, 2017).

However, in practice, this personalization may be unfeasible, due to the staff costs associated with the one-to-one instruction. Therefore, some tools have been proposed to this purpose. Among these, Intelligent Tutoring Systems (ITS) stand out due to its high interactivity and personalized instruction. ITS simulate a personalized tutor that follows student progress, identifies his current learning level, and provides personalized feedback (suggestions or explanations) (Herder, Sosnovsky, & Dimitrova, 2017).

ITS have been used mainly for autonomous learning. However, these ITS can also be used to support face-to-face instruction (Nye, 2015). In the latter case, ITS is used to reinforce the main concepts, thus contributing to improve learning quality and decreasing school dropout. The ITS based interventions largely depend on the teachers as they guide, motivate, and follow-up on the students. However, despite the relevance of the instructors, little attention has been paid to classroom teachers. This situation has been identified as one of the most important barriers to the implementation of ITS (Nye, 2014). Another important challenge for the adoption of ITS, is the low level of ICT literacy among teachers. This might be a reason for the low motivation of teachers to participate in ITS projects (Kozma & Vota, 2014).

This article describes a framework based on intelligent tutoring systems for blended learning. The framework includes some elements to include the teacher in the process. Likewise, a prototype was developed and tested in an elementary school.

The rest of this paper is organized as follows. Section 2 introduces some related works. Section 3 is focused on the proposed framework and its main components. Section 4 describes the prototype developed. Section 5 presents a case study. Finally, Section 6 concludes and discusses areas of future work.

2. Related work

Steenbergen-Hu & Cooper (Steenbergen-Hu & Cooper, 2013) presented a meta-analysis of the effectiveness of ITS on college students. The authors argue that ITS can be used as an independent or complementary learning tool. Moreover, the authors highlight the important role of teachers and pedagogy in the ITS-based learning. The active participation of the teachers is fundamental to the success of ICT-based learning processes (Conde & Soto, 2012).

VanLehn (VanLehn, 2011) carried out a state-of-the-art study on both human-guided tutoring and ITS, in this study human tutors are shown to be more efficient, followed closely in effectiveness by ITS. The author defines eight elements that should be included in tutoring systems: (i) More detailed and accurate assessments and diagnoses of students; (ii) Selection of individual tasks; (iii) Sophisticated tutoring strategies; (iv) Control of the interaction with the learner; (v) Knowledge of the domain; (vi) High student motivation; (vii) Feedback from students; (viii) Cooperation between student and tutor; (ix) Tutoring strategies according to students behavior.

Nye (Nye, 2015) carried out a systematic review of the literature between 2009 – 2012 aimed at analyzing the possible barriers to ITS adoption. The results indicate that more attention must be paid to teachers, as the ITS adoption requires communication between teachers, students, and parents. Moreover, teacher’s motivation is a key factor in the adoption and effectiveness of these learning environments. Olney et al (Olney et al., 2012) introduced an ITS called Guru that seeks to engage students through interactive dialogues. Guru adapts each session based on individual knowledge and provides comments. Furthermore, a comparison between Guru and the human tutoring is presented here, which reports few differences between the two methods.

Alvarez et al., introduced an adaptive environment called MagAdI (Álvarez, Martín, Fernández-Castro, & Urretavizcaya, 2013). MagAdI focuses on integrating knowledge, courses, and teachers. The authors reported increases in the learning gains when the system was used. In addition, there was an increase in student motivation and the teacher’s satisfaction.

Finally, Dolenc and Aberšek (Dolenc & Aberšek, 2015) showed the design and evaluation of an ITS that adapts to the student's cognitive particularities called TECH8. The authors conducted tests with 177 students that show a better performance of TECH8 compared to traditional teaching, but lower performance compared to personalized teaching. The authors conclude that this type of systems can be used as a complement to the traditional classroom teaching.

Literature review allows us to conclude the importance of teachers in the ITS-based learning. Also, it can be concluded that ITS can be used to strengthen the traditional face-to-face teaching method. However, few works can be found that offer modules to follow up on teachers learning.

3. Proposed framework

3.1 Generalized Intelligent Framework for Tutoring (GIFT)

The proposed framework is based on the GIFT Framework. GIFT offers tools, methods, and standards for the creation of ITS (Lane, Yacef, Mostow, & Pavlik, 2013; Sottilare, 2018; Sottilare, Brawner, Goldberg, & Holden, 2012). GIFT provides a modular architecture that includes (Sottilare & Holden, 2013): (1) different tools for ITS development, (2) an instructional manager that incorporates the best practices, and (3) an experimental platform to analyze the components, tools, and methods (Lane et al., 2013).

GIFT is focused on the learner , with the aim of improving the chain of effects of adaptive learning in an iterative way (Sottilare & Holden, 2013). This chain of effects (Figure 1) starts from the understanding of the learner (Student), obtained from their performance, surveys and sensors, which allows determining the learning state more precisely way (Student State in Figure 1). Subsequently, some methods are used to classify student states, improve adaptive capacity, and select the instructional strategies based on those states (Sottilare et al., 2012). The latter leads to a higher probability of influencing learning progress in a positive way by having teaching strategies adapted to the needs of learners (Sottilare et al., 2012).

3.2 Proposed framework

The proposed framework aims at providing a friendly environment for teachers and students. In addition to core functionalities of ITS, the proposed framework includes some functionalities for the evaluation of the teacher knowledge on the framework. Therefore, teacher’s shortcomings can be identified and some hints are shown if necessary. With the latter, the framework is aimed so that teachers can better orient students on the use of the learning tool. The proposed architecture is presented in Figure 2. The main components of the framework are described below.

User Module provides a user interface (student or teacher) to interact with the system. This module uses data from the user´s cognitive status, based on surveys and historical data on the use of the platform. Likewise, this module predicts the future state of the user. The inputs used for the calculation of the state are sources such as performance evaluation (from the domain module) and surveys that have been stored. Since there is no standard learner model for all courses, this module considers a flexible scheme that supports changes according to the requirements (Sottilare, 2012).

Domain Module aims at defining and structuring the knowledge, based on the data obtained from the training environment and the evaluation of the key concepts. The selection of specific knowledge is supported by some design principles of the education system. The following elements are included in the specific domain knowledge: learning objectives, materials, tasks, conditions, performance standards, measures, common misconceptions, and a particular context feedback and a questions library. By reusing existing specific domain knowledge, a significant reduction in development time and cost would be achieved in other contexts (Sottilare, 2012).

Pedagogical Module uses the data from the user module (user state) and domain module (performance evaluation and associated feedback) to determine the order, content, and flow of the instruction. The pedagogical strategies make decisions on the elements of the learning scenario, that is to say, that the pedagogical decisions are made based on the performance and the state of the user. This module influences the training environment to customize the learning.

Tutor user Interface enables teachers and students to carry out tasks such as login, surveys, feedback, post-action reviews, interactive dialogues, review of learning material and, performance assessment, and platform management. To support instructional strategies, the tutor can access learner data from the user model (student or teacher) or data in real time. The tutor – user interfaces may use artificial intelligence techniques to understand and generate natural language responses.

The user Profile integrates historical data on the cognitive state of students and teachers. This information is calculated based on the proposed learning objectives. Likewise, this module includes the data from surveys and the evaluation of the use of the platform (this evaluation appears if the user is a teacher).

Repository of Learning Objects stores the learning objects created by experts. These are retrieved according to the topic to reinforce. Some of these objects are games, SCORM objects, PowerPoint presentations, and simulators.

Course interfaces are created by teachers and include some of the following elements: (i) Learning objectives indicating what the student and teacher will learn, (ii) Relationship of the learning objective to a particular type of knowledge, (iii) Student Assessment based on the particular characteristics.

Connection module connects the Course Interfaces with the Domain Module and Pedagogical Module. Additionally, it is connected to the User Profile to obtain the historical data of students or teachers and to make reports. This module includes additional functionalities that may be required in certain cases, such as the creation of domains and the pedagogical content.

Reports module aims at motivating learners to continue achieving the objectives proposed for each course. To this end, this module shows the students learning progress.

4. Prototype

The framework was used to create a mathematics course for elementary school students in the city of Popayan – Colombia. The prototype seeks an efficient human - computer interaction, by using simple interfaces, allowing a better control of the system by the user. The prototype is composed of the interfaces described below (The figures show the current deployment in Spanish).

A. Main menu

The menu shows the different options for students and teachers. Teachers will have full access to course management, while students have access to their courses.

B. Course management

In this interface, the teachers create, modify or delete courses, based on the deficiencies of each student (Figure 3). The creation of courses involves a set of files for the configuration of the courses. One of the most important files is the domain in which evaluations, instructional strategies (pedagogical module) and status transitions are configured. The Create Course option creates the GIFT domain file based on the information provided by the user. The domain file is an XML file that contains essential information for each course: name, version, description and concepts to study. This file also includes information about course transitions regarding the order of the course and material to be used.

The system may include elements to track the knowledge of the teacher and show help messages or hints if necessary.

C. Student’s interface

In this page, the students find the courses assigned by the teacher in a personalized way to strengthen their knowledge. This interface is provided by GIFT and is shown in Figure 4.

D. Reports

This interface shows the progress achieved by a particular student. These reports make it possible to compare the results in the learning platform, with the results of the activities carried out in the classroom.

5. Case study

For the implementation of pedagogic contents, the area of mathematics was selected, due to its importance for all students. Additionally, from a pedagogical point of view, the learning processes in math use the same mental scheme for all the similar situations, and the knowledge is built from the error so that the platform may stimulate the analysis processes of the students (Castillo, 2008). The created mathematics course allows teaching, practicing and evaluating the basic operations taught in elementary level: addition, subtraction, multiplication, and division (see Figure 5).

Some configuration files are needed to create a course. First, in the domain file, the evaluations, instructional strategies (pedagogical module) and state transitions are configured. This file is part of the domain module and the basis for the creation of any course. After the domain file is created and configured, the teacher can create a course that allows reinforcing in a personalized way the subjects taught in the classroom. For this, it should be assigned a name, a description, if applicable, a survey is attached and, finally, the key concepts are included. Subsequently, the teacher can feed the course using educational resources developed by the expert or by third parties, according to the objectives to be achieved.

The execution of the course begins when the user selects one of the available options. Firstly, the explanation of the concept of each of the operations is provided, giving the foundations necessary to carry out the proposed exercises.

Subsequently, certain practical exercises are proposed to consolidate the theoretical knowledge obtained previously, as shown in Figure 6 (the interfaces maintain the original language). Finally, an evaluation is done to know the failures of the students. The teachers can see the number of successes and failures achieved in each of the operations.

Based on the above, the teacher will obtain a report, which can be compared with the results obtained during classroom exercise and propose personalized reinforcement activities. Based on these results, the teacher will use new learning strategies, making improvements to the course, allowing the learning of the different themes oriented throughout the school year. For the course developed some resources created by third parties and available online.

The creation of future courses by an expert will take into account the particular Colombian context, where the Colombian Ministry of National Education, with the support of some institutions of higher education, proposes that virtual learning objects are constituted mainly by three internal components (Laverde, Cifuentes, & Rincón Rodríguez, 2007) (i) Contents: Types of knowledge with different forms of representation. Definitions, explanations, reading documents, links to other resources, videos, among others, (ii) Learning activities: Orient the student to achieve the learning objectives proposed for a given course, (iii) Context elements: Component that allows the reuse of learning objects in other scenarios.

Some tests are needed to guarantee the quality of the software that must comply with the requirements (Perry, 2007). On the other hand, it is important to analyze aspects such as the way users feel when they use the tool, the usability and the level of acceptance of the tool. The evaluation of the prototype presented in this article takes into account the acceptance of the users in the use of the platform. Although it is important to note that a complete assessment would require measuring the impact of the tool in the learning process, this evaluation will be performed in the future when more courses and learners’ data are collected. First, for the acceptance tests, 7 teachers were selected and contextualized about the advantages of ITS. Likewise, they were instructed on the creation of courses. Special emphasis was placed on low ITC literacy teachers. The survey results are shown in Table 1.

Question	Yes	No
Do you think that this type of platform is a good alternative to support the educational processes of the classroom?	5	2
Do you think that the platform is easy to use?	4	3
Do you find it easy to create courses?	3	4

Furthermore 20 students of 2nd grade were interviewed. Students reacted positively, showing curiosity and interest in using the platform. In general, Students expressed enthusiasm about using the framework. And by questioning them about whether they thought this was a good way to learn, everyone answered positively. The results of the survey are shown in Table 2.

Question	Yes	No
Are you comfortable with the course?	18	2
Do you think this is a good way to learn?	17	3
Accessing the courses is an easy task?	19	1
Is the tool fun?	20	0

6. Conclusions

Teachers play a crucial role in the success of ITS based projects given that his motivation may also contribute to the student’s motivation. Here, the motivation of teachers is addressed by the inclusion of some functionality for the teacher learning. The framework includes some elements to help teachers use the framework. The framework supported by an ITS is tested in a Blended learning course. The preliminary tests of usability allow glimpsing its acceptance.

Future work will concentrate on including intelligent features from the ITS area in SPOC courses (Cook, Kay, & Kummerfeld, 2015; Walkington & Hayata, 2017). Furthermore, another area for future work is the use of the framework in Flipped classroom teaching (Han, Lin, Li, & Ju, 2017).

References

Álvarez, A., Martín, M., Fernández-Castro, I., & Urretavizcaya, M. (2013). Blending traditional teaching methods with learning environments: Experience, cyclical evaluation process and impact with MAgAdI. Computers & Education, 68, 129–140.

Castillo, S. (2008). Propuesta pedagógica basada en el constructivismo para el uso óptimo de las TIC en la enseñanza y el aprendizaje de la matemática. Revista Latinoamericana de Investigación En Matemática Educativa, 11(2), 171–194.

Conde, D. C. R., & Soto, Á. P. G. (2012). Modelo de acción docente con medios informáticos y telemáticos. Pixel-Bit: Revista de Medios y Educación, (40), 151–170.

Dolenc, K., & Aberšek, B. (2015). TECH8 intelligent and adaptive e-learning system: Integration into Technology and Science classrooms in lower secondary schools. Computers & Education, 82, 354–365. https://doi.org/10.1016/j.compedu.2014.12.010

Han, G., Lin, M., Li, C., & Ju, J. (2017). Flipped classroom teaching design based on SPOC in ordinary undergraduate college. In 2017 12th International Conference on Computer Science and Education (ICCSE) (pp. 617–620). IEEE. https://doi.org/10.1109/ICCSE.2017.8085566

Herder, E., Sosnovsky, S., & Dimitrova, V. (2017). Adaptive Intelligent Learning Environments. In Technology Enhanced Learning (pp. 109–114). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-02600-8_10

Hwang, G.-J., Sung, H.-Y., Hung, C.-M., Huang, I., & Tsai, C.-C. (2012). Development of a personalized educational computer game based on students’ learning styles. Educational Technology Research and Development, 60(4), 623–638. https://doi.org/10.1007/s11423-012-9241-x

Kozma, R. B., & Vota, W. S. (2014). ICT in Developing Countries: Policies, Implementation, and Impact. In Handbook of Research on Educational Communications and Technology (pp. 885–894). New York, NY: Springer New York. https://doi.org/10.1007/978-1-4614-3185-5_72

Lane, H. C., Yacef, K., Mostow, J., & Pavlik, P. (Eds.). (2013). Artificial Intelligence in Education. Memphis: Springer.

Laverde, A. C., Cifuentes, Y. S., & Rincón Rodríguez, H. Y. (2007). Toward an instructional design model based on learning objects. Educational Technology Research and Development, 55(6), 671–681. https://doi.org/https://doi.org/10.1007/s11423-007-9059-0

Nye, B. D. (2014). Barriers to ITS Adoption: A Systematic Mapping Study. In Intelligent Tutoring Systems (pp. 583–590).

Nye, B. D. (2015). Intelligent Tutoring Systems by and for the Developing World: A Review of Trends and Approaches for Educational Technology in a Global Context. International Journal of Artificial Intelligence in Education, 25(2), 177–203. https://doi.org/10.1007/s40593-014-0028-6

Olney, A. M., D’Mello, S., Person, N., Cade, W., Hays, P., Williams, C., … Graesser, A. (2012). Guru: A computer tutor that models expert human tutors. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7315 LNCS, 256–261. https://doi.org/10.1007/978-3-642-30950-2_32

Perry, W. E. (2007). Effective Methods for Software Testing: Includes Complete Guidelines, Checklists, and Templates. John Wiley & Sons.

Sottilare, R. A. (2012). Considerations in the development of an ontology for a generalized intelligent framework for tutoring. U.S. Army Research Laboratory - Human Research & Engineering Directorate, 19–25.

Sottilare, R. A. (2018). A Comprehensive Review of Design Goals and Emerging Solutions for Adaptive Instructional Systems. Technology, Instruction, Cognition & Learning, 11(1), 5–38.

Sottilare, R. A., Brawner, K. W., Goldberg, B. S., & Holden, H. K. (2012). The Generalized Intelligent Framework for Tutoring (GIFT). Orlando, FL: US Army Research Laboratory--Human Research \& Engineering Directorate (ARL-HRED), (October), 1–12.

Sottilare, R. A., & Holden, H. K. (2013). Recommendations for Authoring , Instructional Strategies and Analysis for Intelligent Tutoring Systems ( ITS ): Towards the Development of a Generalized Intelligent Framework for Tutoring ( GIFT ). AIED 2013 Workshops Proceedings, 7, 150.

Steenbergen-Hu, S., & Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. Journal of Educational Psychology, 105(4), 970–987. https://doi.org/10.1037/a0032447

VanLehn, K. (2011). The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems. Educational Psychologist, 46(4), 197–221. https://doi.org/10.1080/00461520.2011.611369

Walkington, C., & Hayata, C. A. (2017). Designing learning personalized to students’ interests: balancing rich experiences with mathematical goals. ZDM, 49(4), 519–530. https://doi.org/10.1007/s11858-017-0842-z

Wentzel, K. R., & Brophy, J. E. (2014). Motivating Students to Learn. Taylor and Francis.

Framework inteligente para Blended Learning

An Intelligent framework for Blended Learning

Resumen

Abstract

1. Introduction