COU e-Learning Meeting Notes Thursday 16 August 2007

From OUCEL Wiki

Contents 1 Course Management Systems 2 Kirkpatrick model of training 3 metrics of organizational impact 4 PLIITS 4.1 Jeanne, Windsor 4.2 David, Waterloo 4.3 Aldo, Guelph 5 Letter to VPA should include

[edit] Course Management Systems

• Bart Collins, PhD

bcollins@purdue@edu

• How you can use measurement strategies to feed back into your CMS e-Learning strategy
• 6000 classes, 3200 courses use the WebCT Vista platform
• 2 – 3 classes have a WebCT Vista Presence
• 95% are F2F
• They are an ITunes U institution
• They have some licensing systems to run Windows Software in a web browser plug-in. Software Remote.
• Purdue has a number of IT staff, lots of money supporting them, are they harming the learning environment, or inappropriately using of them. Do we need all of them?
• Have had a renaissance of resources in the last few year in order to buy them…
• Kirkpatrick’s model of training: do people like the service to does the behaviour improve the organization
• Is the knowledge being transferred into daily behaviour?
• I have to like it to learn it, then learn it to change my behaviour?
• How can you approach a strategy to address these areas?
• Is CMS use related to student success?
• How does our CMS or other eLearning Tools impact the learning process?
• Graduation rate: we want to produce students who transition to the workforce effectively
• Data often comes in too late to be able to evaluate
• And it is not good enough: classes are too large >300 students and it’s hard to know the students and find the successful from unsuccessful students
• The Student: Do I have the ability to study the material? / Do I have the time?
• Pulled the data from the students: with the registrar’s assistance, and TLT at Purdue, John
• Not a huge difference in Verbal SAT scores between the regular standing students and the ones on academic probation
• Math SAT’s: no difference: there is a difference at the high end with the Dean’s List students but not a huge difference at the low end
• Kim Holland: there would not be a huge sample size in the high end and low end students
• Differences in CMS logins: the Dean’s list students log in more than average students, and more so than the average students
• WE ARE NOT SUGGESTING THAT CMS LOGINS MEANS THAT IT IS IMPROVING LOGIN.
• Linda: correlation does not imply causation
• Bart: It is an index of engagement in course materials
• Starts to predict the GPA of students as a function of course management system logins
• Aldo: Can you look at classroom attendance against the same data: however that data is not there, except when clickers are used, because then the attendance can be tracked.
• If I know your course activity from the course management system, I can predict with 80% accuracy whether you are going to get a D or F or A, B or C.
• 3 times more predictive than SAT scores
• The more activity on the course, the better the prediction: Distance Learning courses can be used
• Can tell 3 weeks into the semester where the student may fail
• Surveyed student and faculty users of the systems, and whether each system supports the seven good guides to undergraduate education
• Students like Podcasting more than the Faculty, and think it emphasizes time on task
• Clickers: prompt feedback, but students hated the clickers in terms of faculty/student contact. Students are not feeling the love

[edit] Kirkpatrick model of training

1. Reaction- do people like the service?
2. Learning- does it facilitate learning?
3. Behaviour- does learning impact behaviour?
4. Results- does it improve the organization?

[edit] metrics of organizational impact

• student graduation rate (retention problem)

[edit] PLIITS

Purdue Learning Impact of IT systems scale

• contact
• active learning
• student-student cooperation
• prompt feedback
• time on task
• high expectancy
• diverse learning

[edit] Jeanne, Windsor

• Implementation of Sakai. Paid the $10k membership for the implementation.
• RSmart – company that assists with implementations of the software
• Cheaper than Blackboard, but there are costs to implementing the software
• Distance courses, Blended Courses
• Implemented in Winter, and all Summer Courses e-courses were moved to Sakai
• By next fall 2008, almost everyone would be migrated to the new application
• Vicki is the home-grown system at Windsor.
• Have some projects using the gradebook
• CLUW – Collaborative Learning Environment Windsor
• Not easy to export data from old systems, or a high cost to exit from old systems
• Sakai said it is easy to export the content into another system
• No course kits are allowed to be developed into the new system.

[edit] David, Waterloo

• Was involved in home-grown system and Angel
• Angel Administrator.
• First course in 1995. HTML only.
• Home grown system based on T5: Teams, Tasks, Tools and Tutoring
• Migrated away from Blackboard and moved to Angel: attractive: semi-open, allowed for customization
• Moved LMS to central IT support, and wanted the faculty to have a consistent look and feel with the first system they had on campus
• May 2004: launched Angel in the following year. Angel 5.6. Some missing tools and they subcontracted some of these out.
• Fall of 2004: Survey of the faculty to see how they were doing with the software: 10% response rate. Communication tools were very good. Many did not like the gradebook tool but has improved in the new version.
• 90% of the instructors would recommend Angel to a colleague
• Run distance education specific surveys
• Appreciate the video and audio delivery, interactive SCORM utilities
• Representatives that provide pedagogical support for each faculty office and distance education department
• Angel support them with an online ticket system, listserv and some phone support
• SIS – integrated wth Peoplesoft for the course list, the class list is not populated until the request is made for the course. Download and upload into the Peoplesoft system
• Interest in adding professional development courses.
• Desire for a stable system: tried to leave the system as is and gone through the vendor for support
• 730 courses per term, up from 320 two years ago, and from the initial 30. (1600 courses per semester)
• 2,000 instructors, 23,000 students (?), increase in number of concurrent sessions. As many as 3200 concurrent sessions
• CMS has required more support over time: more students and faculty using it
• More advanced applications being requested
• More faculty warming up to the technology, and following the students in terms of technology skills (?)
• Angel has an ePortfolio component: testing for the last two years

[edit] Aldo, Guelph

• Blackboard and D2L
• Using Blackboard since 1999, D2L since 2001
• D2L worked with the office of Open Learning at Guelph to develop D2L. 50% of D2L is based in that office at Guelph
• D2L – 245 DE courses. 18,000 enrollments
• 1000 courses in Blackboard – on-campus, 1000 designers
• You cannot differentiate between faculty and others they just have “Designer Access”
• Some people are using Blackboard for departmental community building exercises
• BB CE 6 completed by Sept 2006. Got to the limit of BB CE 4, and ran out of capacity – built on original versions of WebCT: performance on the web server was getting worse, and that version was not scalable
• Cost jumped from small amount of money to a lot of money. More resources required to support 6.
• Running three application servers, one Oracle DB server with load balancing, $100,000 over the original cost
• Done some data mining on the usage patterns: difficult to keep track of what everyone is doing with their courses due to the proliferation of users
• Trying to quantify what is being used in specific courses: how are people using the tools in the CMS
• Survey of Blackboard Use: Most courses using between 4 and 6 tools
• Top used tools: Syllabus, Calendar, Grade Book, My Grades, Announcements, Web Links
• Rating of Courses and the navigation, organization and aesthetics of the course: Most courses rated average 78%, and 9% were above average, but none were “superior”
• Aldo with share the criteria: subjective ratings: how many clicks to get to the content, how many icons on the homepage. Rated by the staff member
• 76% of courses use the standard template
• Instructional Richness: amount, kind of content, how much discussion. 58% rated moderate
• Bill: academic integrity: limits the popularity of the assessment tool.
• Respondis: Quiz question tool. Guelph has a site-license for it.
• No student presentation tool: now have to buy an ePortfolio tool
• Interface: it is getting worse in terms of the overall interface. Too many clicks for specific objects.
• D2L cons: high licensing cost
• Trends: Inertia vs. pain. One-stop shopping versus best of breed
• Some of the tools built into the CMS are second rate compared to the other tools out there: The blog tool in the CMS is not actually really very good.
• http://edutools.info
• http://its.queensu.ca

Queen’s U: the community said that D2L had a much better pedagogical model, but there was not enough impetus for change: the cost for time, hiring, hardware was far too high.

• Cost and complexity of systems forces the IT to take control

• ACTION ITEM: Denise: will be providing a draft of the letter prior
• ACTION ITEM: Forward my notes to the group
• ACTION ITEM: Gwen will take care of everything with regard to the evaluation (Helen) report
• ACTION ITEM: York University to host the LISTSERV
• ACTION ITEM: Next Meetings: Regional Groups and Summer Institute
• ACTION ITEM: Submit the case studies, data, and other materials to the assessment and eval group by the end of September.
• ACTION ITEM: Teleconference meeting every two months on day and time to be determined.
• Put in the briefing note to the VPA’s
• ACTION ITEM: F2F meeting: Educause? (Seattle) OCUL (Oct 18/19)? EDC meeting? McGraw-Hill Ryerson Conference 17 is the CDC (STLHE),18 of November. Investigate this for the time frame. Mostly likely to be November 16th
• ACTION ITEM: Next Summer Institute: Week of August 11th: Likely to be 12, 13, 14. Recommend that it will continue to be in the GTA.
• Bring others to the Summer Institute: twice the size. What would it look like as a larger panel
• Linda: if we get too big, will we get any work done.
• Denise: is this information sharing or information doing.
• ACTION ITEM: Web presence: Need to sort out the name with COU/John/Jan. With Wiki. We will come up with a name. Ontario Universities Council on e-Learning (?).
• Patrick: We’re about building community. And there is a year that passes without the collaboration and deep interaction.
• Need to establish ourselves as a community outside of this room.
• Web presence: needs to be a way of sharing resources. Can we collect that information and post it along to other members
• Should we be investigating group licensing?
• We should hook up communities of practice around the technology areas: iTunes U
• Who is using what?
• Financial: Pedagogy but not the admin
• Gwen: group licensing. Invested a lot in coordinating this, but CIO’s are looking at this anyway
• ACTION ITEM: Richard, Gwen and Andrew: follow up with feedback about the Summer Institute
• Jack: Study to determine the extent that e-Learning is being adopted by Ontario Universities: need one or two people to look at Jack’s study. Develop a document to do the study we really want.
• Karen: 17 pages of notes from dinner last night.
• Bill: learning object about the 10 commandments for learning in undergraduate education
• Future of CLOE?
• Denise: Only Brock has withdrawn. Interest in making a decision about it.
• Jack and some people at UofT will be engaging in it
• How many are withdrawing or waiting for slow death?
• Preference to wind down participation in CLOE
• How many are going to EDUCAUSE?
• MUIRHEAD COMMISSION: how many drafts will we see?
• OCAV learning objectives: ALDO?
• When will know when the VPA meetings are?

[edit] Letter to VPA should include

• thank you
• here's what we did
• here's what we'd like to do
• here's what I'd like you to do