MOCLog for administrators and study program managers

Introduction

Overview of MOCLog

Procedures for operations/menus

Introduction

You are an administrator for Moodle,
and are asked:

  • How many courses are utilized productively and in which departments;
  • what percentage of courses use further interactive tools beyond the distributed course materials, or, more specifically,
  • which courses in what departments employ the quiz tool.

Maybe you are recording such statistics over a longer period of time and want to document the usage of Moodle-programs over several semesters.

With the MOCLog Dashboard, you, as the administrator, have a tool at hand that allows you to compile excerpts of Moodle-log data necessary for usage statistics of a Moodle installation.

MOCLog enables you to compile data documenting the usage of single tools or pre-defined tool groups, not only in a single program but across an entire program category.

Thus, you can assess the interactivity of Moodle courses within an faculty department and follow up on its development over time.

You are a study program manager,
and you want to get an idea of the usage of Moodle among your instructors and students:

  • To what extent do instructors/students use the interactivity options?
  • Which tools are used by which instructors/students, and to what degree? Or more specifically, e.g.:
  • Which instructors/students employ the quiz tool?

Maybe you are recording such statistics over a longer period of time and want to document usage over several semesters.

With the MOCLog Dashboard, you, in the role of Moodle “Administrator” or “Manager”, have a tool at hand that allows you to compile extracts of Moodle-log data necessary for compiling the usage statistics of a Moodle installation.

to the top

Overview of MOCLog


MOCLog in Moodle

The MOCLog application can be downloaded from the following internet address: http://sourceforge.net/p/moclog/code/5/tree/trunk/, and implemented in Moodle as a normal plug-in with the following configuration parameters:

MOCLog uses Moodle log data to generate statistics and visualisations of user activity, development of learning platform tool usage over time, as well as aggregated user and tool activities.
To calculate the aggregated data, the log data are read from the “mdl_log” table at each iteration of the cron job and aggregated in “mdl_block_moclog_log” on a daily basis.
This table saves the user data in a daily aggregated format. Data in this table are recorded continually and are not modified or deleted.

Overview of MOCLog Functionality

Questions can be evaluated according to the degree of tool usage within a course area, such as:

  • Data related to specific courses: to what degree were Moodle tools like Quiz, Assignment, Wiki etc. accessed in each course?
  • Data related to specific tools: to what degree were Moodle tools like Quiz, Assignment, Wiki accessed within a group of courses in total?

Two concrete examples may help to illustrate the value of Moodle by running MOCLog to evaluate the data of a faculty department.

Example 1:

In the study area “General Education” of the Technology and Computer Science Department of Bern University of Applied Sciences, at present (spring semester 2012), 137 Moodle courses are offered along with the on site studies.

Of these, at the moment,

  • 62 courses are used only for the distribution of teaching materials.
  • 49 courses additionally use one of the further learning activities, such as Assignment, Forum, Quiz and Wiki.
  • 26 courses use the collaborative and interactive tools of the learning platform intensively – two or more of the Assignment, Forum, Quiz and Wiki learning activities.

Example 2:

According to the evaluated Moodle access statistics, the tools utilised in the study area xy, besides the materials for distribution (55%), mainly include the Forum (10%), Assignment (15%) and Quiz (13% of courses) tools.

MOCLog can be configured specifically for each evaluation category. Items to be set specifically:

  • selection of courses from a course category
  • report type (short list, tool list, data export)
  • selection of tools
  • type of access (view/upload etc.)
  • type of visualisation (data/graphics)
  • user group to be evaluated
  • period of observation

to the top

Procedures for operations/menus


The following part of the MOCLog – “Instructions for Administrators and Study Program Managers”, is guided by the structure of the “Start Page” that opens after clicking on the dashboard.

At the bottom of the Start Page, you will find three menu items; these settings must be configured before performing an analysis with MOCLog.

Instructions for menu settings are discussed in detail in the “Settings” chapter.

Dashboard

The “Dashboard” can only be accessed by Moodle “Administrators” and “Managers”.

As a manager, you will require either global or course-specific rights.

After clicking on the Dashboard, a screen will open that hereinafter will be referred to as the “Start Page”; courses may vary depending on the learning institution.

The Administrator is able to see all course categories and to select from the entire dropdown list. The Manager will only see those categories for which he or she has assigned this role. In the example shown here, the course category “HKB University of the Arts Berne/test environments” from the dropdown list is displayed, together with the courses in this category.

Selection of Courses – Course Category

In the course list of the selected category, individual courses can be disabled/enabled via the checkbox to the left of each course.

Pressing the button “View Report” will display the respective reports.

Below, each dashboard dropdown menu is discussed individually.

Report Types – Administrator

As an Administrator, you can request or have listed the following data:

  • Course Activity = selected courses with their number of hits on each Moodle tool,
  • Tool Activity = each tool with the total number of hits across all courses,
  • or Raw Data Export = .csv file export.

Report Types – study program manager

According to the differing requirements for the role of a study program manager, a menu different from that of the learning platform administrator will be presented to you. You can request or have partially listed the following data:

  • Course Activity = selected courses with their number of hits on each Moodle tool,
  • Teacher Facilitation Report = number of hits on each Moodle tool per Instructor,
  • Student Learning Report = number of hits on each Moodle tool per student,
  • or Raw Data Export = .csv file export.

Figures

The figures shown in the following sample reports all begin with:

  • the absolute number of hits on each tool,
  • all respective users in the selected courses,
  • the entire preset observation period,
  • the preset observation period calculated per day.

For comparison of courses with different numbers of users, absolute numbers can be apportioned to a standard group size of 20.

This will influence the above mentioned report types: “Course Activity” and “Raw Data Export”.

“Start Page” menus in detail:

Course Activity

The report type “Course Activity” displays the number of users per each of the selected courses (first column). In the columns to the right, a summary of hits (Total) and the number of hits per tool follow.

Clicking the “View Report” button displays a report using the presets outlined below, as follows:

In “Total”, all hits on a course are summarised. The class size is not taken into account (see “Normalisation“).

In the above example, the 7 users of the “Testkurs Literarisches Schreiben” (“Test course Creative Writing”) had a total of 128 hits; among these, 40 were on Wiki, 8 on course resources, labels and folders, and 24 on quiz and assignment. Hits may have occurred via the different actions “view”, “update”, “delete” and “upload/write”.

For relative comparisons between courses or course groups, the following option may be enabled:

By enabling this option, the number of hits will be apportioned to a standard class size of 20 and displayed as shown below:

Note the differences, e.g. in the column “Total”:

In the normalised version, courses with the same total in absolute numbers have different relative values according to the number of users. Hence, a ranking of the courses in the “Total” column will result in a different order as compared to the ranking in absolute numbers. In this case, the normalised value is calculated by dividing the absolute numbers by 7 (real users) and multiplying by 20 (norm users).

Tool Activity (Administrator only)

The report type “Tool Activity“ displays the number of hits (usages) for each tool by the selected roles within the defined period among the selected courses. In the above example, there are 108 hits on quizzes in the selected courses. The respective results per user are shown in parentheses.

Further down, the evaluation is illustrated in a bar and a pie chart.

Graph #1 shows the absolute hit numbers in a bar chart.

In graph #2, they are displayed as a pie chart of relative values.

The visualisation type can be selected in the configuration settings “Report Types”.

In this example, the prevailing activity was the hits on Wiki with 180 hits (=31% of all hits), followed by 108 (19% of all hits) on Forum and Quiz.

Teacher Facilitation Report – Student Learning Report (Study program manager only)

These two reporting options offer an overview of the usage of each tool/all selected tools by a faculty (students).

Note: For tool selection, see “Activity Profile Settings” configurations.

All users of the selected courses with a Teacher role are displayed along with their respective count of hits on each tool.

The column “Total Score“ shows the cumulative hits (in absolute numbers) on the listed tools, counted across all courses in which the user (here, Teacher) is enrolled.

In the above example, there were 48 hits by the first user, 15 of these on Wiki, three on course resources, labels and folders, and 9 on Quiz and Assignment. Hits may have occurred via the different actions “view”, “update”, “delete” and “upload/write”.

Raw Data Export

Selected data can be exported in .csv file format for further processing, e.g. in Excel.

Timeframe

The evaluation period is defined by the settings on the Start Page.

For comparison purposes, this can be narrowed down, e.g. to a period of intense usage.

Selecting a checkbox will enable a feature, after which the data may be customized.

Roles

The selection of the group to be evaluated can be narrowed down by

their roles, e.g. to students, if you wish to assess only the hits by students.

User groups

In “User Groups“, you can select by default either “All Users” or particular groups. Groups are defined in “User Group Settings” (see instructions below).

Note: User groups are visible to and editable by all administrators. Therefore, they can also be deleted by any other administrator!

Note: Instructors and students are only distinguished if their roles (see above) are selected!

Profile Settings

In Profile Settings, you can select pre-defined profiles. Profiles are defined in “Profile Settings“ (see instructions below).

Among these profiles, you can select a configuration for the “activity profile”. The profile determines the weighting of each tool (see instructions below) and the method of visualisation.

Normalisation

If this checkbox is selected, the number of hits will be apportioned to a group size of 20 users. This allows for the comparing of different courses.

With this checkbox unchecked, you will have to interpret the absolute numbers displayed. Group size will play a role as, of course, given the same user behaviour, a larger group will result in higher hit counts.

Note: Concerning observation period normalisation, all figures are apportioned to one day (per day), even with different observation period settings (see Timeframe).

Settings

The three menu items on the Start Page offer settings for:

  • selection of tools
  • type of access (view/upload, etc.)
  • type of display (data/graphics)
  • user group to be evaluated

Activity Profile Settings – Selection of Tools for the Report

Here, you can “Load” or “Delete” an existing profile, or create a new one by selecting “Create New Profile”.

Select “Default activity profile” and click “Load” – you will then see these settings:

Here all default values are set to 1; i.e. all tools and access types are equally weighted.
You can now create a new profile with adjusted values (different weighting).

Note: Not all access types are available for each tool – e.g., students are not entitled to “delete” the tool “folder”. Reports only display the sum of the different actions “view”, “update”, “delete” and “upload/write”.

Click “Create New Profile” and enter a name for the new profile, e.g. “assessment”.

Adjust the weightings to your requirements.

In the example, only hits on the two tools “assignment” and “quiz” will be evaluated.
Moreover, with this setting, all hits on quiz will be double-counted.

Subsequently, data of all users of selected courses will be retrieved from Moodle log data according to the other settings and totalled (in absolute numbers or apportioned to the default group size of 20).

For the activity profile, also consider “Other Settings”:

Here you can exclude certain elements, e.g. all elements not available to students, at the course level as well as the resources level (hidden), from being counted.

Complete with “Save”. The following will then be shown:

On clicking “Go back” on the following screen, a new activity profile will be displayed in the drop-down list.

Report Setting Profile/Type of Visualisation (Data/Graphics)

In Report Settings, you can design your report based on the selected “activity profile“ (see above).

In addition to the selected “activity profile”, you can choose the type of visualisation (graphic #1 = bar chart, graphic #2 = pie chart, data table). In Report Settings, the presets for the “activity profile” can be combined with the type of visualisation (bar chart, pie chart, data table).

The graphic visualisation is essential for a “tool activity report”. You are not able to influence other report types.

The procedure in detail:
Here, you can “Load” or “Delete” an existing profile, or create a new one with “Create new Profile”.

Click “Create profile settings”.

The following screen allows you to select the activity profile (the previously defined activity profile “assessment” is shown here).

Below, select the desired Graphics and save them with “Save”.

By clicking on “Go back” in the following screen, the newly created profile is available in the pull-down menu “Profile Settings” on the Start Page.

User Group Settings

You are interested in a cross-comparison of a group of students in different courses.

In “User Group Settings”, you can select a group of individuals.
The procedure to create a new group is analogous to the procedure for “activity profile settings” or “profile settings“ (see above) – with the functions “Load” and “Delete” for existing user groups and “Create user group” to create a new one.

The procedure in detail:

Click “Create user group” and enter a name for the new “user group”; in this example: “USG-3″.

After saving this information click “Add users to this group” on the following screen.

Two windows will be displayed. In the right window, all users of the Moodle environment are listed for selection in alphabetical order, while in the left window, the users shown are those that you have added using “Add”.

Mark the users to be selected in the right window and click “Add” to move them into the “Selected Users” window on the left.

Click “Save” and “Go back” as shown on the following screen.

The newly created user group “USG-3″ can now be selected in the pull-down menu “User Groups”.

“View Report” will generate a report with the preset configuration.

to the top

Comments are closed.