Assessing E-Learning Systems Success In Nigeria An Application Of The DeLone And Mclean Information Systems Success Model
ABSTRACT
Aim/Purpose
This study is based on the DeLone and McLean
s Information Sy stems Success
(D&M ISS) model which was modified to determine the success factors
responsible for the accep t ance of an e learning system called Canvas by
students of a Nigerian University.
Background
The adoption of eLearning has been
under studied withi n the context of
developing countries. There have been calls in the literature for further research
from a developing country perspective. This study attempts to fill this
knowledge gap by investigating the fa ctors responsible for students adoption
of eLe arning in a Nigerian University.
Methodology
The study was built on the premise that system quality (SQ), service quality
(ServQ) and information quality (IQ) are determinants of behavioral intention
to use Canvas and user satisfaction of Canvas, both o f which in turn influence
the actual usage of Canvas. Responses from 366 students were analyzed with
AMOS 22 using structural equation model (SEM) to test the relationships
between the constructs of the proposed model.
Contribution
The study contributes
to the research gap about the paucity of studies in the
adoption of eLearning in developing countries that have placed emphasis on
the use of eLearning systems or on the software quality attributes of the
systems under investigation.
Findings
The results
partially supported the effect of the quality antecedents on
behavioral intention and user satisfaction of students. There was full support
for the relationship between behavioral intention and user satisfaction of
students on their actual usage of Canvas.
Recommendation
s
for Practitioners
Recommendation for
Researchers
The level of significance of the constructs identified in the study
will guide the
formulation of strategies and methods t hat could further enhance the adoption
of eLearning systems in de veloping countries, specifically Nigeria.
Theoretically, the study contributes to the body of knowledge on eLearning
adoption by empirically validating the DeLone and McLean model in a different
context, specifically sub Saharan Africa.
Future Research
Fu
ture research could investigate the factors that influence instructors
behavioral intentions to use eLearning applications and also the effect that the
instructors have on students adoption of eLearning.
Keywords
eLearning systems, IS success, universit
ies, Nigeria
CHAPTER ONE
The growth of the internet and advances in information and communication technologies (ICT) has impacted almost every aspect of life. The education sector is one of these areas where technology has made a massive impact. The processes involved from when a student applies for admission up till graduation can be captured by computer applications. The internet and the web facilitate these processes irrespective of geographical locations. Students’ usage of social media and the internet have ensured that educationists change the traditional methods used to teach and learn. As such teaching and learning have also been transformed. The use of learning management systems and the web enables the access to learning resources without any boundaries of time or location.
ELearning is a product of the evolution of the learning process due to the application of ICT in the classroom. According to Benninck (2004), eLearning is the use of technology such as the internet, web applications and computing equipment to facilitate teaching and learning. The benefits of eLearning to students and instructors cannot be overemphasized. ELearning enhances the teaching and learning process, it provides access to education curriculum, reduces the long-term costs of learning, and through distance learning, it expands educational opportunities (Lwoga, 2014). Studies have shown that eLearning is an effective method of instruction and increases knowledge (Lwoga, 2011; Salter, Karia, Sanfilippo, & Clifford, 2014). ELearning has also been found to motivate students’ interaction and ease communication (Arkorful & Abaidoo, 2015).
Despite the global penetration of eLearning systems, only a few Nigerian higher education institutions have embraced eLearning. In developed countries, eLearning is well established due to the enormous amount of resources that have been invested in the education sector. Developing countries, like Nigeria, are still at an infancy stage, and this is mainly due to the unique challenges faced by higher education institutions. Inadequate infrastructure, lack of funding, overcrowded classrooms and the dearth of instructors are cited as some of the challenges faced by educational institutions in Nigeria (Boyi, 2014). These generic challenges, identified by Boyi (2014), can be extended to the implementation of technology (eLearning) in the classroom in Nigerian universities. For example, lack of funding will hinder the purchase and implementation of technology in Universities, the training required by the instructors and technical support. Mtebe and Raisamo (2014a) also highlighted the cost of acquiring, managing, and maintaining ICT infrastructure as a challenge to adopting and implementing eLearning in sub-Saharan Africa. They also stated that these costs can be significantly reduced by implementing cloud versions of the eLearning applications. While this is true, the challenge of students and instructors accessing the internet for course material will pose a bigger threat to the adoption of eLearning as the students and instructors will be responsible for internet and mobile device costs.
In order to ensure the success of eLearning technology implementation, there is a need to identify the factors that contribute towards the adoption of eLearning by students of Nigerian Universities. So far there has been very little research carried out to understand the factors responsible for students’ adoption of eLearning in Nigeria, and this could be due to the relatively small number of higher education institutions that have the facilities and resources to implement eLearning. One notable study on the acceptance of eLearning in a Nigerian University was done by Yakubu and Dasuki (2018). The
research employed the UTAUT framework to investigate the factors responsible for influencing the students’ usage of the Canvas learning management system (LMS). The UTAUT (unified theory of acceptance and use of technology) framework, by Venkatesh, Davis, and Davis (2003), was formulated to explain users’ behavioral intentions to use technology based on four fundamental constructs namely: performance expectancy, effort expectancy, social influence, and facilitating conditions. These four constructs were responsible for influencing use behavior through behavioral intentions. Yakubu and Dasuki (2018) called for further research using other frameworks to investigate Nigerian students’ acceptance of eLearning.
In an attempt to answer the call posted by Yakubu and Dasuki (2018), this study uses a modified version of the updated DeLone and McLean information systems success model (D&M ISS) as a theoretical lens to identify the factors responsible for the acceptance of eLearning by Nigerian students. The study aims at examining the system and information design attributes and their relationship to use of eLearning technologies via behavioral intention and user satisfaction. The eLearning application under investigation in this study is called Canvas LMS. Canvas LMS is used to organize courses using a unified calendar and syllabus system, facilitate discussion forums, plagiarism checks, and to set up and grade assignments and quizzes. The findings from this study will provide a theoretical foundation and empirical evidence explaining the determinants responsible for the successful usage of Canvas LMS by students of a Nigerian higher education institution.
L
LITERATURE ITERATURE RREVIEWEVIEW
The original DeLone and McLean’s Information Systems Success (D&M ISS) model (DeLone & McLean, 1992) was formulated to identify the factors responsible for defining information systems success. Six factors were identified, as shown in Figure 1. System quality and information quality were responsible for influencing use and user satisfaction of information systems. Use and user satisfaction influence each other and they both have an influence on individual impact which in turn influences organizational impact.
Figure 1. DeLone and McLean IS Success Model (DeLone & McLean, 1992)
DeLone and McLean revised the original model (see Figure 2) in response to the strengths and weaknesses identified by researchers. The impact variables (individual and organizational) were grouped as net benefits. A new variable, service quality was added to the model to capture the importance of service as a contributor to IS success. Finally, the “use” construct was subdivided into 2, intention to use (attitude) and actual use (resultant behavior).
Figure 2. Updated DeLone and McLean IS Success Model (DeLone & McLean, 2003)
The independent constructs employed by the model, i.e., information, system and service quality, are referred to as the quality antecedents. They measure the quality of the software or application being investigated. System quality measures the ease-of-use, functionality, reliability, flexibility, data quality, portability, integration, and importance of the system (DeLone & McLean, 2002). Information quality measures the accuracy, timeliness, completeness, relevance, and consistency of the information output of the system. Service quality measures the tangibility, reliability, responsiveness, assurance, and empathy of the service provided by the system. These three exogenous variables were hypothesized to influence intention to use and user satisfaction of the system. The net benefits construct captures the overall impact of the system on the users of the system and was identified as the most important of the success factors by DeLone and McLean (2003). If the net benefits of using a system are positive, then there is a higher chance that the system will be used as well as an improved user satisfaction.
The updated D&M ISS model was developed to be used in the context of e-commerce but it has also been used in several ways to measure different types of information systems. The D&M ISS model has been extended by adding new constructs to the model. For instance, Hsu, Chang, Chu, & Lee (2014) extended the model by adding the trust variable in the context of e-commerce. Similarly, Jagannathan et al. (2018) employed the model by adding security as a construct in a study to examine the acceptance of internet banking. Other authors have focused on validating the model in e-commerce (DeLone & McLean, 2004), Knowledge Repository Systems (Qian & Bock, 2005), eGovernment systems (Wang & Liao, 2008), and a mandatory information system (Iivari, 2005).
Results from studies that have used the D&M ISS model have produced mixed results. Applying the model to mandatory information system, Iivari (2005) found that system quality and information quality were predictors of user satisfaction but they had no effect on system use; while Jagannathan et al. (2018) observed that for an internet banking application, system quality had no effect on user satisfaction but information quality influenced user satisfaction. The mixed results imply that the model can assist in identifying the success variables in different settings as shown in the results obtained by Iivari and Jagannathan. In the case of the internet banking application, security and information quality are expected to be the determining factors for IS success which was the case in Jagannathan’s study and in the mandatory setting system quality and information quality would have little effect on the use as it is mandatory to use the system irrespective of the quality antecedents..
In the context of eLearning, the updated D&M ISS model has been applied to different types of systems. Lin (2007) proposed a model to examine the determinants for the successful use of online learning systems by undergraduate students. The results indicated that system quality, information, quality, and service quality significantly influenced use via behavioral intention and user satisfaction. Lwoga (2014) employed the model to examine the factors that predict usage of a web-based learning management system by students. Lwoga’s research was based on prior work (Cheng, 2012; Lee, Yoon, & Lee, 2009), which introduced a new construct, “instructor quality” that was found to significantly influence perceived usefulness of an eLearning system. Cheng (2012) used constructs from the technology acceptance model (TAM) (Davis, 1989) and the updated D&M ISS model (Delone & McLean, 2003) to propose a model to examine the effect of quality antecedents on learners’ intention to use an eLearning system. Lwoga’s results showed that system quality was a determinant of user satisfaction this also corresponded to the results of Cheng (2012) and Chen (2010). Information quality and service quality had no relationship with user satisfaction, which is an unexpected finding and contradicts the findings of Cheng (2012) and Chen (2010).
R
RESEARCH ESEARCH MMODELODEL
This study adopts the six variables used in the updated DeLone and McLean IS success model (D&M ISS model) to determine the factors responsible for the usage of Canvas LMS by students of a private university in Nigeria. The updated D&M ISS model was chosen as the basis for this study as it has been tested and validated previously, especially in the domain of elearning adoption (see Lin, 2007; Lwoga, 2014; Mohammadi, 2015; Mtebe & Raisamo, 2014b). In Nigeria, Ajoye and Nwagwu (2014) used a conceptual model adapted from the updated D&M ISS model to investigate the effect of the quality antecedents on user satisfaction of the postgraduate school portal for the University of Ibadan. The results from Ajoye and Nwagwu (2014) showed that the quality antecedents significantly influenced user satisfaction thus justifying the investment in the school portal. Further support for this claim, in the context of eLearning in Nigeria, is the need to assist other higher education institutions aiming to adopt eLearning systems. Another reason for selecting the D&M ISS model is in order to meet the objectives of this study, which is to examine the system and information design attributes of Canvas LMS. The D&M ISS model is a natural fit because it captures both the information and system qualities, unlike other acceptance theories that only cater for a subset of these attributes. For example, the “perceived ease of use” construct used in the technology acceptance model (TAM) by Davis (1989) only captures how easy
it is to use the technology, which is only one out of six attributes that measure system quality according
to Jalote (2008). The six attributes as identified by Jalote (2008) are functionality, reliability, usability,
efficiency, maintainability, and portability. The system quality construct in the D&M ISS model measures
most of these attributes depending on the context it is used in.
This study is based on a modified version of the updated D&M ISS model similar to the models applied
by Lin (2007) and Mohammedi (2015) which were both used to measure the success of eLearning. The
modification is in order to fit the context of eLearning systems. The proposed conceptual model shown
in Figure 3 will be used to help identify the factors that influence the adoption and use of eLearning by
students of higher education institution in Nigeria. The quality antecedents (information quality, system
quality, and service quality) derived from the updated D&M ISS model will be used to explain the effects
they have on the actual usage of Canvas LMS through user satisfaction and behavioral intention.
System Quality
Actual Usage
Behavioral Intention
to use Canvas LMS
User Satisfaction
Service Quality
Information Quality
Figure 3. Conceptual Model
The proposed model (Figure 3) makes the assumption that system quality, information quality, and
service quality are expected to have a positive influence on user satisfaction and behavioral intention to
use Canvas LMS, which in turn will influence the actual usage of Canvas LMS. These foundations lead to
the following set of hypotheses:
SYSTEM QUALITY (SQ)
SQ measures the desired characteristics of the eLearning system (Lwoga, 2014). These characteristics
include usability, responsiveness, adaptability, and reliability. Previous studies have shown that SQ has
an influence on user satisfaction (Chen, 2010; Freeze, Alshare, Lane, & Wen 2010; Mohammadi, 2015;
Ramayaha, Ahmad, & Lo, 2010) and behavioral intentions (Bhuasiri, Xaymoungkhoun, Zo, & Rho, 2012;
Lin, 2007; Mohammadi, 2015; Ramayaha et al., 2010). Consequently, this study hypothesizes that:
Hypothesis 1a: System quality will have a positive influence on user satisfaction with Canvas LMS.
Hypothesis 1b: System quality will have a positive influence on behavioral intention to use Canvas LMS.
I
INFORMATION NFORMATION QQUALITY UALITY (IQ)(IQ)
In this study, IQ is referred to as the quality of the course resources uploaded to the Canvas LMS as evaluated by the students. The quality attributes include timeliness, availability, easy to comprehend, relevance, completeness, and security. Previous studies have shown that IQ has a positive influence on user satisfaction of eLearning systems (Freeze et al., 2010; Mohammadi, 2015; Ramayaha et al., 2010) and behavioral intention to use eLearning systems (Bhuasiri et al., 2012; Lin, 2007; Mohammadi, 2015; Ramayah & Lee, 2012). It is hypothesized that the quality of the course content will be a determinant of students’ intention to use the eLearning system and their overall satisfaction with using the system for learning, thus:
Hypothesis 2a: Information quality will have a positive influence on user satisfaction with Canvas LMS.
Hypothesis 2b: Information quality will have a positive influence on behavioral intention to use Canvas LMS.
S
SERVICE ERVICE QQUALITY UALITY (S(SERVERVQ)Q)
Service quality which is similar to the facilitating conditions construct in the unified theory of acceptance and use of technology (UTAUT) model (Venkatesh et al., 2003) encompasses the support rendered by either the developers of the system or a support team such as the IT department. The characteristics of ServQ in this study include knowledge, empathy, responsiveness, and effectiveness of the software support unit of the University whose responsibility is to support the eLearning system. Similar to previous studies, it is hypothesized that ServQ will influence user satisfaction (Freeze et al., 2010; Mohammadi, 2015; Ramayaha et al., 2010) and behavioral intentions (Mohammadi, 2015; Ramayaha et al., 2010). Thus:
Hypothesis 3a: Service quality will have a positive influence on user satisfaction with Canvas LMS.
Hypothesis 3b: Service quality will have a positive influence on behavioral intention to use Canvas LMS.
U
USER SER SSATISFACTION ATISFACTION (US)(US)
User satisfaction is an individual’s perception of being satisfied with the system relative to the individuals’ initial expectations of using the system. US characteristics captured in this study include dependability, relevance, usefulness, and effectiveness. User satisfaction of eLearning systems has been shown to influence behavioral intentions (Lin, 2007; Mohammadi, 2015) and actual use (Mohammadi, 2015).
Hypothesis 4: User satisfaction will have a positive influence on behavioral intention to use Canvas LMS.
Hypothesis 5: User satisfaction will have a positive influence on the actual use of Canvas LMS.
B
BEHAVIORAL EHAVIORAL IINTENTION NTENTION (BI)(BI)
Behavioral intention is the likelihood of an individual to engage in a particular behavior. In the updated D&M ISS model, all the quality constructs were deemed to influence BI. In the context of eLearning, several studies have found support for the relationship between BI and actual usage of eLearning systems (Lin, 2007; Mohammadi, 2015). Thus:
Hypothesis 6: Behavioral intention to use Canvas LMS will have a positive influence on the actual use of Canvas LMS.
A
ACTUAL CTUAL UUSAGE SAGE (AU)(AU)
Actual usage measures the actions of an individual using the eLearning system. Similar to UTAUT, the updated D&M ISS model postulates that the behavioral intention of an individual leads to the actual behavior. Lin (2007) and Mohammedi (2015) have corroborated this in their studies on eLearning systems as earlier mentioned.
M
METHODOLOGYETHODOLOGY
The study aims at investigating the success factors responsible for the students’ usage of an eLearning application called Canvas LMS. Canvas LMS allows students to access course resources, participate in group discussions, submit assignments, and take online quizzes and tests. The use of Canvas LMS (in the observed university in this study) is mandatory for all students as all the courses are loaded on to the Canvas platform at the beginning of the semester. Based on the objectives of the study as well as the characteristics of the population (i.e., users of Canvas LMS), purposive sampling technique was used. The target population is all registered undergraduate students of the selected private university in Nigeria. The university has used Canvas LMS for over 6 years and all instructors and students are trained on the use of Canvas LMS as part of their orientation process. Similar to previous studies using the D&M ISS model, a quantitative method was adopted. Data was collected using an online survey. An online survey was the preferred method of collecting data in order to minimize errors that might be caused by data handling, increase the rate of response and reduce costs.
The survey instrument was divided into three sections (see Appendix for survey questions). Section one consisted of a consent form to assure and ensure confidentiality of the respondents. Section two of the survey focused on demographic data of the participants and experience of using Canvas. Section three contained statements regarding the factors used in the conceptual model. These statement questions were derived and validated from previous studies on eLearning success and adoption (Lin, 2007; Mohammadi, 2015). Four items were used to measure the system quality variable; they included reliability, availability, ease of use, and functionality. Information quality was measured through comprehension, accuracy, relevance, and ease of access. Service quality measured the empathy, dependability, knowledge, resources, and response time of the Canvas support staff.
The students’ age was measured in years and grouped as follows: (Under 20), (20-29), (30-39) and (40 and above). Gender was coded as 1 and 2 which represented male and female gender respectively. The values 1, 2 and 3 were used to measure how long a student has used Canvas where 1= less than 1 year,
2= 1-2 years and 3= over 2 years. A five-point Likert scale ranging from strongly agree (1) to strongly disagree (5) was used to measure all six variables used in the model in Figure 1. A pilot test was performed among 21 students to ensure that there were no typo errors and that the measurement instrument was easy to comprehend. Minor changes were made to the survey questions based on the feedback from the pilot test.
An email was sent out to all undergraduate students of the university that contained the link required to respond to the survey. The email also informed the students of the research objectives stating that participation was completely voluntary. The responses were anonymous and confidential in order to protect student’s privacy. Also, the students could end the survey at any time or choose not to answer questions they were not comfortable with.
A two-step approach was used to analyze the data obtained from the survey as suggested by Anderson and Gerbing (1988). SPSS statistics 21.0 and AMOS version 21.0 was used to analyze the data using structural equation modeling (SEM) technique. Reliability and validity of the measurement model were tested via confirmatory factor analysis (CFA) and the hypothesis was tested using the structural model.
D
DATA ATA AANALYSISNALYSIS
D
DEMOGRAPHIC EMOGRAPHIC CCHARACTERISTICS OF THHARACTERISTICS OF THE E RRESPONDENTSESPONDENTS
Table 1 shows the characteristics of the respondents. There were a total of 383 responses. After data screening, which involved the removal of incomplete or unengaged responses, 366 valid responses remained. Of the 366 responses, 44% were female while the remaining 56% were male. In terms of experience, of using Canvas, the majority of the respondents (77.1%) have used Canvas for over 1 year while only 22.9% have used the application for less than a year. Almost half the respondents (48.1%) were under 20 years of age, 42.3% were between 21 and 29 years old, 5.5% were under 40 but over 30 years of age while 4.1% were above the age of 40. Thus 90.4% were under the age of 30.
Table 1. Respondents Demographic Data (n=366)
DEMOGRAPHIC CHARACTERISTIC
FREQUENCY
PERCENT (%)
GENDER
FEMALE
161
44
MALE
205
56
EXPERIENCE
UNDER 1 YEAR
84
22.9
1-2 YEARS
147
40.2
OVER 2 YEARS
135
36.9
AGE
UNDER 20
176
48.1
21-29
155
42.3
30-39
20
5.5
ABOVE 40
15
4.1
S
SCALE CALE VVALIDATION AND ALIDATION AND MMEASUREMENT EASUREMENT MMODELODEL
Table 2 shows the construct measurement items used in the survey as well as a summary of the descriptive analysis for each of the variables employed by the research model. With the exception of service quality (ServQ), the analysis shows positive responses for all the constructs as their respective means are below the value of 2. To establish the internal reliability of the variables used in the model, Cronbach’s alpha for the observed responses was measured. The values for Cronbach’s alpha were found to be over 0.8 which is good (George & Mallery, 2003) for all the constructs except AU and US, which were over 0.9 and is deemed as excellent according to George and Mallery (2003).
To be consistent with similar studies, structural equation modeling was applied using a two-stage process. Step 1, the measurement model, measures the hypothetical constructs based on the observed variables while step 2, the structural model, describes the relationships between the 6 constructs. The maximum-likelihood method was used to estimate the parameters of the measurement model and then the goodness-of-fit test was carried out on the measurement model. To confirm a good fit between model and the corresponding data, two items (ServQ2, and IQ1) were removed from the service quality and information quality constructs respectively.
Table 2. Descriptive analysis of the constructs
CONSTRUCT
MEAN
SD
CRONBACH’S ALPHA
MEASUREMENT ITEMS
System Quality (SQ)
1.57
0.755
0.809
SQ1. Canvas is available most of the time
SQ2. Overall Canvas is highly reliable with minimal downtime
SQ3. The response time of Canvas is very good
SQ4. Canvas is easy to use
CONSTRUCT
MEAN
SD
CRONBACH’S ALPHA
MEASUREMENT ITEMS
Information Quality (IQ)
1.76
0.991
0.893
IQ2. Overall, the information I get from Canvas is easy to understand
IQ3. Overall, the information I get from Canvas is relevant for my studies
IQ4. Overall the information within Canvas is secure
Service Quality
(ServQ)
2.10
0.959
0.896
ServQ1. The Canvas support staff have the resources required to ensure Canvas is available and usable at all times
ServQ3.Canvas support staff gives me prompt service by responding quickly to my request for help
ServQ4.The Canvas support staff have the knowledge to support my requests when needed
ServQ5.The Canvas support staff shows empathy towards me while responding to my requests
User Satisfaction (US)
1.78
0.921
0.913
US1. I am satisfied with the functions of Canvas
US2. Canvas has eased the study processes for me
US3. I am generally satisfied using Canvas
US4. I am satisfied with the support I get in using Canvas
Behavioral Intention (BI)
1.60
0.905
0.884
BI1. I intend to use Canvas frequently this semester
CONSTRUCT
MEAN
SD
CRONBACH’S ALPHA
MEASUREMENT ITEMS
BI2. I am willing to encourage other people to use Canvas
BI3. I predict I will use Canvas frequently in the coming semesters
BI4. It is a good idea to use Canvas
Actual Usage (AU)
1.53
0.848
0.902
AU1. I use canvas frequently
AU2. I use many of the functions in Canvas
Table 3 displays the recommended and actual measurement model values for the different fit indicators as suggested by Hu and Bentler (1999), Kline (2015) and Hair, Black, Babin, Anderson, and Tatham (2010). All indices are within the recommended values as shown in Table 3 indicating a good model fit.
Table 3. Model Fit indices
(CFI: Comparative fit index, GFI: Goodness-of-fit index, AGFI: Adjusted goodness-of-fit; RMSEA: Root mean square error of approximation; RMSR: Root mean square residuals; NFI: Normed fit index.)
FIT INDEX
RECOMMENDED VALUE
MEASUREMENT MODEL
STRUCTURAL MODEL
CFI
>0.90
0.956
0.956
GFI
>0.90
0.907
0.907
AGFI
>0.80
0.877
0.879
RMSEA
<0.08
0.059
0.058
RMSR
<0.10
0.038
0.039
NFI
>0.90
0.924
0.923
To evaluate the measures used in this study, validity tests were carried out on the constructs, convergent validity and discriminant validity tests were implemented in order to validate the scales. To verify convergent validity, Hair et al. (2010) recommend 3 conditions that must be met: 1) a CR value of over 0.8, 2) an average variance extracted (AVE) should exceed 0.5, and 3) the AVE should be less than
the corresponding CR. Table 4 shows that all the items measured have a CR value over 0.8, all the AVE values were found to be greater than 0.5, and finally, the AVE values were all found to be less than their corresponding CR values thus establishing convergent validity.
Table 4. Composite Reliability, Convergent Validity, Discriminant Validity and Factor Correlation Matrix
To assess discriminant validity, Hair et al. (2010) recommend that the ASV (average shared variance) and the MSV (maximum shared variance) should be lower than their corresponding AVE value; similarly, the square root of the AVE must be higher than the correlation values (Bagozzi & Yi, 1988). Table 4 also shows that discriminant validity conditions have been met for all the constructs based on these 3 conditions. As all the constructs have been tested for convergent validity and discriminant validity, the next step in the data analysis was to evaluate the structural model by examining the research model and testing the hypothesis.
H
HYPOTHESIS YPOTHESIS TTESTING AND ESTING AND SSTRUCTURAL TRUCTURAL MMODELODEL
Table 3 (model fit indices) shows that the fit measures for the structural model are all within the recommended (Hair et al., 2010; Kline, 2015) values which indicate the data fits well with the model.
CR
AVE
MSV
ASV
IQ
SERVQ
US
BI
SQ
AU
IQ
0.895
0.741
0.368
0.160
0.861
SERVQ
0.898
0.689
0.349
0.090
0.168
0.830
US
0.916
0.732
0.349
0.090
0.148
0.591
0.856
BI
0.890
0.670
0.325
0.130
0.570
0.081
0.089
0.818
SQ
0.818
0.531
0.368
0.140
0.607
0.169
0.161
0.469
0.728
AU
0.906
0.828
0.078
0.05
0.227
0.143
0.236
0.280
0.225
0.910
Service Quality
Actual Usage
R2= 0.12
Behavioral
Intention
R2= 0.35
User Satisfaction
R2= 0.35
Information
Quality
System Quality
0.58***
.05
0.02
0.46***
0.20**
-.04
0.21***
0.27***
0.01
Figure 2: SEM analysis showing path coefficients, significance and R-square
(Notes: *** indicates: p < 0.0001 and ** indicates: p < 0.01)
In order to test the hypothesis and the structural model, confirmatory factor analysis was used to
evaluate the path’s significance and strength. The strength of the relationship between the exogenous
and endogenous variables (R2) was also measured. Figure 2 and Table 5 show the findings from the
confirmatory factor analysis.
Table 5: Structural model and hypothesis testing
(NS: non-significant relationship, S: a significant relationship)
HYPOTHESIS PATH ESTIMATE P SIGNIFICANCE
H1a US <— SQ 0.075 0.446 NS
H1b BI <— SQ 0.257 0.004 S
H2a US <— IQ 0.02 0.759 NS
H2b BI <— IQ 0.399 *** S
H3a US <— ServQ 0.575 *** S
H3b BI <— ServQ -0.031 0.576 NS
H4 BI <— US 0.009 0.867 NS
H5 AU <— US 0.182 *** S
H6
AU
<—
BI
0.259
***
S
The path coefficients indicate that of the 9 relationships tested, only 5 were found to be significant. H1b, H2b, H3a, H5, and H6 were all supported while there was no support for H1a, H2a, H3b, and H4. All the quality antecedents with the exception of service quality significantly influenced behavioral intentions of students to use Canvas which is the exact reverse as observed in the relationship between user satisfaction and the quality antecedents, where only service quality was found to significantly influence user satisfaction. User satisfaction was found to influence actual usage but was non-significant in its relationship with behavioral intentions. The final relationship is between, behavioral intentions and actual usage and this was found to be significant.
D
DISCUSSIONISCUSSION
This study uses the DeLone and McLean IS success model (Delone & McLean, 2003) to examine the success factors responsible for the acceptance of Canvas, a learning management system, by students of a Nigerian private university. In an attempt to explain the students’ acceptance of Canvas, the model used six of the factors: service quality, system quality, information quality, user satisfaction, behavioral intention, and actual usage.
System Quality (SQ) was hypothesized to influence both behavioral intention (BI) to use Canvas and user satisfaction (US) of Canvas. This is consistent with similar studies (Mohammadi, 2015; Ramayaha et al., 2010). The findings from this study only supported the relationship between SQ and BI where SQ was found to have a positive and significant influence on the students’ behavioral intention to use Canvas. This result indicates that students’ intention to use canvas is influenced by availability, reliability, response time, and design functionalities of Canvas. The non-significant relationship between SQ and US indicates that student’s satisfaction is not based on the quality on the system, which is unexpected and contrary to prior studies (Lwoga, 2014; Mohammadi, 2015; Ramayaha, et al., 2010). As most of the students believe that the quality of the Canvas application is good (mean of SQ=1.57), a possible explanation for the non-significant relationship might be the effect of the instructors on students using Canvas. In the studied environment, all instructors must publish their courses on Canvas. The instructors have varying experience and perceptions about the Canvas application, which results in different ways that they engage their students. And because each student can take up to four courses per semester (i.e., 4 different instructors) their responses with regards to user satisfaction might be affected. Another possible reason for the non-significant relationship is the fact that over 77% of the students have used Canvas for more than one year. Their experience in using Canvas might indicate that other factors would be responsible for their satisfaction other than the system quality.
The DeLone and McLean IS success model suggests that information quality (IQ) has an influence on both BI and US. In this study, the results show that IQ was statistically significant to BI which corroborates other studies (Ramayaha et al., 2010). This implies that the students find the information they get from using Canvas is complete, accurate, easy to comprehend, and relevant to their studies. There was no support and evidence to show that a relationship exists between IQ and US. This is similar
to the findings by Lwoga (2014). Lwoga attributed this to the students being unfamiliar with a newly introduced system. In this study, the non-significant relationship between IQ and US is probably due to the effect of the instructors, as mentioned above, where the student’s responses might differ based on the type of instructor they are basing their responses on.
Service quality (ServQ) was proposed to significantly influence BI and US. Unlike IQ and SQ, ServQ’s relationship with US was found to be statistically significant while the relationship with BI was non-significant. In the relationship between ServQ and US, the positive influence of the service quality on students’ satisfaction with using Canvas shows that the IT support (technical staff knowledge, empathy, and response) and the IT resources (internet access, computers, and server availability) are determinants of user satisfaction. These findings are found to be consistent with other studies (Cheng, 2012; Li, Duan, Fu, & Alford, 2012; Ramayaha et al., 2010). On the other hand, ServQ was non-significant in its relationship with BI which is similar to the findings of Lwoga (2014). Lwoga attributed this to the participants’ moderate experience of using the system, thus the need for technical support does not influence students behavioral intention. In this study, over 77% of the student’s responses indicate usage of the system for over 1 year and as such the findings in this study can be credited to the participating students’ experience.
Like the DeLone and McLean IS success model (Delone & McLean, 2003) user satisfaction (US) was postulated to influence both actual usage (AU) and behavioral intention (BI). The findings in this study support the relationship between US and AU which means that the usage of the Canvas is positively influenced by the students’ satisfaction of using the application. This corroborates the findings by Mohammadi (2015) and Lwoga (2014). Surprisingly there was no support for the relationship between US and BI. This indicates that there is no association between student’s satisfaction and their behavioral intention to use Canvas. This is contrary to prior studies (Chang, 2013; Islam, 2012; Mohammadi, 2015), and the reason for the non-significant relationship might be attributed to the mandatory use of Canvas by all students.
Prior acceptance models have shown that Behavioral intention (BI) translates to actual usage (AU). The technology acceptance model (Davis, 1989), the theory of planned behavior (Ajzen, 1985), the unified theory of acceptance and use of technology (Venkatesh et al., 2003), as well as the DeLone and McLean IS success model (Delone & McLean, 2003) were built on the premise that behavioral intention results in actual behavior. The findings in this study are consistent with the mentioned theories as well as other studies in the context of eLearning (Mohammadi, 2015; Olatunbosun, Olusoga, & Samuel, 2015; Raman, Don, Khalid, & Rizuan, 2014).
C
CONCLUSIONONCLUSION
There is a lack of literature pertaining to students’ attitude towards the adoption of eLearning technologies in Nigeria. This is partly due to the fact that a majority of the higher education institutions have not yet implemented any form of eLearning systems. The need to understand the variables that will enable the adoption of eLearning by students is therefore very important as it will guide institutions willing to implement the use of eLearning technologies for instruction.
The results from this study have shown that out of the nine proposed relationships; only 5 were supported as shown in Table 5. The relationship between software quality and information quality on behavioral intention was significant. Similarly, the effect of service quality on user satisfaction was statistically significant. And finally, user satisfaction and behavioral intentions on actual usage were also significant hence the 5 hypotheses were all supported.
Information quality was found to have the strongest influence on the behavioral intention of students to use Canvas followed by the system quality. Higher education administrators and educators should ensure that the eLearning system is reliable, easy to use, and always available. The administrators should also ensure that instructors are trained on how to use the system effectively for teaching so as to ensure that the information/resources placed on the system are relevant, comprehensible and accurate. Service quality had an effect on students’ behavioral intentions; this is possibly due to the fact that most of the students are well acquainted with the system. Thus there should be adequate training on using the system and increased support of the eLearning system, especially for novice users.
In the relationship between the quality antecedents and user satisfaction, only service quality was found to influence user satisfaction also user satisfaction had no influence on the behavioral intention of students’ actual usage of Canvas. This showed that the system quality and information quality were not responsible for students’ satisfaction in using Canvas. User satisfaction, in this study, captures the students’ perception of being content in using the system. Thus the service quality has an impact on the students’ satisfaction. The quality of the information was non-significant to user satisfaction and significant to behavioral intentions probably due to the fact that students had no choice on whether to use the system or not, in a mandatory setting the information quality should, therefore, be relevant to the learning outcomes of the course. While the students’ response to system quality showed that the system was of good quality, it still was not enough to influence user satisfaction. This is possibly because the use of the system was to fulfill learning outcomes as opposed to hedonic reasons. Behavioral intention and user satisfaction were both statistically significant in their relationship to actual usage though behavioral intention had a stronger effect when compared to user satisfaction.
R
RESEARCH AESEARCH AND ND PPRACTICERACTICE IIMPLICATIONSMPLICATIONS
Developed countries are proficient in the application of technology in the classroom as prior studies have directed the efforts required to enhance eLearning in these countries. Meanwhile, developing countries are still playing catch up. In countries like Nigeria, the majority of higher education institutions has not yet implemented technology usage in the classroom. The presence of cultural differences between developed and developing countries means that the factors that contribute to the use of eLearning applications must be investigated.
There have been limited studies, carried out in Nigeria and perhaps Africa as a whole, that have tried to understand how the quality of an application contributes to the use of a system especially in the context of eLearning. This study examines the quality antecedents of technology, i.e., system, information and service quality, and their relationship with behavioral intention and actual usage of Canvas learning
management system. The findings from this study have substantial practical and theoretical implications with regards to the adoption of eLearning in developing countries, especially in Nigeria.
The significant levels of each variable used in this research help to guide Nigerian university administrators and educators on the determinants to place emphasis on when implementing eLearning in their institutions. For example, administrators can arrange workshops to assist instructors on the best practices in designing course structure and content so as to improve students learning outcomes and participation by using the eLearning system. The universities should also ensure that the applications they implement are easy to use, reliable, and provide the relevant information to aid the students learning process. There should also be adequate support and training not just for instructors but also for students which will improve the adoption of eLearning by the students. While several implications were identified by the findings of this study, there are also some limitations, as discussed in the subsequent section.
In theory, the Delone and McLean IS success model, when applied to eLearning systems in a mandatory setting, has shown that user satisfaction is not a determinant of behavioral intention. Also, the system and information quality attributes do not determine the student’s satisfaction of using the system. Service quality has no impact on the behavioral intention of students to use the eLearning system, especially for experienced users.
L
LIMITATIONS OFIMITATIONS OF SSTUDY ATUDY AND ND FFUTURE UTURE WWORKSORKS
The student sample used in this study was selected from a small private university in Nigeria whose teaching style is based on the American curriculum, thus the findings from this study should not be generalized to reflect the eLearning adoption by the entire student population in Nigeria. Future research, therefore, should focus on a wider scope to include public universities and moderators such as culture, gender, and socio-economic factors, in order to form a model that is generally more applicable. Also, this study focuses only on a specific eLearning technology, i.e., a learning management system (LMS); hence students’ perceptions might be different when adopting other eLearning technologies such as Moodle and Blackboard.
The context of this study was in a mandatory setting, which was believed to affect some of the hypothesized relationships. To really understand student’s perception of eLearning tools future research should consider students perception to adopt eLearning in a context were usage is voluntary. This study was conducted using quantitative methods which employed the use of an online survey to gather responses. These responses are limited by the students’ ability to reliably remember their perceptions towards the use of Canvas as well as their willingness to sincerely self-report. The use of a mixed method in future studies could provide more insights on the students’ adoption of eLearning applications.
The study also omitted the net benefits construct as proposed by DeLone and McLean (2003) which would have been adapted to capture the benefits of eLearning to the students. Future research should consider the inclusion of this important success factor. Finally, this study concentrated on the adoption of eLearning by students without any consideration to instructors input into Canvas or their adoption to
the eLearning system. eLearning applications, like Canvas LMS, are heavily dependent on the usage and input by the instructors. Future research could investigate the factors that influence instructors’ behavioral intentions to use eLearning applications and also the effect that the instructors have on students’ adoption of eLearning.
R
REFERENCESEFERENCES
Ajoye, M. B., & Nwagw
Ajoye, M. B., & Nwagwu, W. E. (2014). Information su, W. E. (2014). Information systems ystems uuser ser ssatisfaction: A atisfaction: A ssurvey of urvey of tthe he ppostgraduate school ostgraduate school portal, portal, UUniversityniversity of Ibadan, Nigeria. of Ibadan, Nigeria. Library Philosophy and PracticeLibrary Philosophy and Practice, , ppaper 1192aper 1192..
Ajzen, I. (1985). From intentions to actions: A theory of
Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. In planned behavior. In J. Kuhl & J. Beckmann (Eds.), J. Kuhl & J. Beckmann (Eds.), Action Action controlcontrol (pp. 11(pp. 11–39). Berlin Heidelberg: Springer.39). Berlin Heidelberg: Springer. https://doi.org/10.1007/978https://doi.org/10.1007/978–33–642642–6974669746–3_23_2 Anderson, J. C., & Gerbing, D. Anderson, J. C., & Gerbing, D. W. (1988)W. (1988). . Structural equation modeling in practice: A review and recommended Structural equation modeling in practice: A review and recommended twotwo–stepstep approach.approach. Psychological Psychological BBulletinulletin, 103, 103(3), 411.(3), 411. https://doi.org/10.1037/0033https://doi.org/10.1037/0033–2909.103.3.4112909.103.3.411
Arkorf
Arkorful, V., & Abaidoo, N. (2015). The role of eul, V., & Abaidoo, N. (2015). The role of e–learning, advantageslearning, advantages,, andand disadvantages of its adoption in higher disadvantages of its adoption in higher education. education. International Journal of Instructional Technology and Distance Learning, 12International Journal of Instructional Technology and Distance Learning, 12((11), ), 2929–4242..
Bagozzi, R. P., & Yi, Y. (1988). On the evaluati
Bagozzi, R. P., & Yi, Y. (1988). On the evaluation of structural equation models. on of structural equation models. Journal of the Academy of Marketing Journal of the Academy of Marketing Science, 16Science, 16(1), 74(1), 74–94.94. https://doi.org/10.1007/BF02723327https://doi.org/10.1007/BF02723327
Benninck, R. (2004). Implementing
Benninck, R. (2004). Implementing elearningelearning from the corporate pefrom the corporate perspective.rspective. Online Knowledge Tree eOnline Knowledge Tree e–Journal. Journal. RetrRetrieievedved fromfrom http://en.wikieducator.org/images/8/83/Bennink.pdfhttp://en.wikieducator.org/images/8/83/Bennink.pdf
Bhuasiri, W., Xaymoungkhoun, O., Zo, H., & Rho, J. J. (2012). Critical success
Bhuasiri, W., Xaymoungkhoun, O., Zo, H., & Rho, J. J. (2012). Critical success factors for efactors for e–learning in developing learning in developing countries: A comparative analysis countries: A comparative analysis betweenbetween ICT experts and facultyICT experts and faculty.. Computers & Education, 58Computers & Education, 58(2), 843(2), 843–855.855. https://doi.org/10.1016/j.compedu.2011.1https://doi.org/10.1016/j.compedu.2011.10.0100.010
Boyi, A. A. (2014). Education and sustainable national development in Nigeria:
Boyi, A. A. (2014). Education and sustainable national development in Nigeria: CChallenges and way forward. hallenges and way forward. International International Letters of Social and Humanistic Sciences, 14Letters of Social and Humanistic Sciences, 14, 65, 65–72.72. https://doi.org/10.18052/www.scipress.com/ILSHS.14.65https://doi.org/10.18052/www.scipress.com/ILSHS.14.65
Chang, C. C. (2013). Exploring the determinants of e
Chang, C. C. (2013). Exploring the determinants of e–learning systems continuance intention in academic libraries. learning systems continuance intention in academic libraries. Library Management, 34Library Management, 34(1/2), 40(1/2), 40––55.55. https://doi.org/10.1108/01435121311298261https://doi.org/10.1108/01435121311298261
Chen, H. J. (2010). Linking employees
Chen, H. J. (2010). Linking employees’ ’ ee–learning system use to their overall job outcomes: An empirical study based on learning system use to their overall job outcomes: An empirical study based on the IS success model. the IS success model. Computers & Education, 55Computers & Education, 55(4), 1628(4), 1628–1639.1639. https://doi.org/10.1016/j.compedu.2010.07.005https://doi.org/10.1016/j.compedu.2010.07.005
Cheng, Y. M. (2012). Effects of quality antecedents on e
Cheng, Y. M. (2012). Effects of quality antecedents on e–learning acceptance. learning acceptance. Internet Research, 22Internet Research, 22(3), 361(3), 361–390.390. https://doi.org/10.1108/10662241211235699https://doi.org/10.1108/10662241211235699
Davis, F. (
Davis, F. (1989). Perceived us1989). Perceived usefulness, perceived ease of use, and acceptance of information technology. efulness, perceived ease of use, and acceptance of information technology. MIS QuarterlyMIS Quarterly, , 1313(3), (3), 319319–340.340. https://doi.org/10.2307/249008https://doi.org/10.2307/249008
DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable.
DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Information Systems Research, 3Systems Research, 3(1), 60(1), 60–95.95. https://doi.org/10.1287/isre.3.1.60https://doi.org/10.1287/isre.3.1.60 DeLone, W.DeLone, W. H., & McLean, E.R. (2002). Information systems success revisited.H., & McLean, E.R. (2002). Information systems success revisited. Proceedings of the Proceedings of the Hawaii International Hawaii International Conference on System SciencesConference on System Sciences ((HICSSHICSS)), , 29662966–2976. 2976. https://doi.org/10.1109/HICSS.2002.994345https://doi.org/10.1109/HICSS.2002.994345
DeL
DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model oone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: Af information systems success: A tenten–year year update. update. Journal of Management Information Systems, 19Journal of Management Information Systems, 19(4), 9(4), 9–30.30. https://doi.org/10.1080/07421222.2003.11045748https://doi.org/10.1080/07421222.2003.11045748
DeL
DeLone, W. H., & Mclean, E. R. (2004). Measuring eone, W. H., & Mclean, E. R. (2004). Measuring e–commerce success: Applying the DeLone & McLean information commerce success: Applying the DeLone & McLean information systems success systems success model.model. InternationaInternational Journal of Electronic Commerce, 9l Journal of Electronic Commerce, 9((11), ), 3131–4747..
Freeze, R. D., Alshare, K. A., Lane, P. L., & Wen, H. J. (2010). IS success model in e
Freeze, R. D., Alshare, K. A., Lane, P. L., & Wen, H. J. (2010). IS success model in e–learning context based on studentslearning context based on students’’ perceptionsperceptions.. Journal oJournal of Information Systems Educationf Information Systems Education, 21, 21((22), ), 173173..
George, D., & Malle
George, D., & Mallery, P. (2003). ry, P. (2003). SPSS for Windows step by step: A simple guide and reference.SPSS for Windows step by step: A simple guide and reference. Boston: Allyn and Bacon.Boston: Allyn and Bacon.
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010).
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010). Multivariate data analysis.Multivariate data analysis. (Vol. 5). Upper (Vol. 5). Upper Saddle River, NJ: Prentice Saddle River, NJ: Prentice HHaallll..
Hsu, M. H., Chang, C. M., Chu, K. K., & Lee, Y. J. (2014). Determinants of repurchase intention in online group
Hsu, M. H., Chang, C. M., Chu, K. K., & Lee, Y. J. (2014). Determinants of repurchase intention in online group–buying: buying: The perspectives of DeLone & McLean IS success model and trust. The perspectives of DeLone & McLean IS success model and trust. Computers in Human BehaviorComputers
IF YOU CAN'T FIND YOUR TOPIC, CLICK HERE TO HIRE A WRITER»
