Luke Carrier

Fluid for Moodle

Fluid is a modern take on Moodle's course formats which aims to solve numerous challenges Floream have encountered with the course view in our Moodle platforms. This article will serve as a technical overview of the Fluid plugin framework and will describe the design decisions we made during its development. My colleague Thomas has a great write up on what's wrong with Moodle course formats that I'd recommend reading first.

Where we started

We began by reviewing the tools we already had at our disposal. Moodle content is arranged within a simple schema:

  • Category
    • Category
    • Course
      • Block
      • Block
      • Section
        • Course module (activity or resource)
        • Course module
      • Section
  • Category

A Moodle course using the "topics" course format

Categories can contain child categories and courses, courses are collections of sections and sections are collections of course modules. Each item at each level of this schema has its own "sort order", which allows content creators and administrators some limited control over the presentation of the content.

The course formats shipped with Moodle core and community contributed plugins, with the notable exception of Flexpage, modify the presentation of the sections and activities on the page whilst preserving the uniform structure of the course. This allows course creators to present their content in a grid or with tabs, but it's an all in decision. You can't mix and match layouts to play to the strengths of the content without breaking sections out into different courses, which makes for poor user experience and complex navigation.

Where we are now

A Moodle course using the Fluid course format

A Moodle course using the Fluid course format, configured to use the collapsed course layout and grid section layout throughout. Read on for details of the other layout options we've built.

The challenges

  • For content creators, course formats are just plain inflexible. Moodle courses are expected to have an extremely linear, uniform structure. Whilst this makes for a consistent user experience in many cases, it prevents content creators from selecting the most appropriate presentation for individual pieces of content.
  • Course formats are often abused to house configuration data for themes. Some of our platforms were using the course format to store course-specific settings, such as colours or CSS classes, for use within user-facing blocks and themes when rendering course pages.
  • Many of our forks of community course formats had accumulated a multitude of site-specific hacks features. This feature bloat creates unnecessary cognitive load when sifting through options.
  • Few of our course formats survived a backup and restore, with the majority losing some or all of their configuration.

Furthermore, as a developer, authoring course formats is a complex and daunting task. Developers are expected to consider not only the course view page, but also the necessary changes to the layout to enable editing. Preserving backwards compatibility with older Moodle versions involves some gnarly parameter handling. These design decisions require course formats to duplicate a large chunk of boilerplate code.

Design principles

Provide flexibility in layout

Course formats have grown to conflate three concepts into one:

  • Scheduling units of work (sections), e.g. with the weekly course format.
  • The presentation of activities to users.
  • Storing arbitrary pieces of configuration necessary within the theme.

This issue can likely be attributed to their relative age; the format plugin type dates back to an era in Moodle well before sub-plugins and their scope has grown organically.

Fluid addresses the inflexibility issue by splitting the user interface elements of course formats into two key concepts, each of which is implemented as its own plugin type:

  • Course layouts dictate the presentation of sections within the course view page.
  • Section layouts present activities within these sections. These can be mixed and matched within an individual course, with a default being configured at the course level.

Home configuration data

Fluid provides a third plugin type, metadata, which allows attaching custom configuration options to courses, sections and course modules. Fluid handles the storage, backup and restore of all of this data on behalf of the plugins.

Be bold, favour opinion over options

Options added to the Fluid core or Fluid plugins should have utility across all Moodle sites. All other customisation of the course view, including the presentation of associated metadata, can be delegated to the theme via renderer overrides. Renderers are able to query configuration data added by metadata plugins, allowing for customisation to target specific courses, sections and course modules.

In the same vein, we opted to support Bootstrap 3 exclusively in all of our internally developed course layouts.

Centralise complexity

Do all of the things in Fluid core. By performing the heavy lifting in Fluid itself we can allow plugins to remain focused on the tasks they're being authored to accomplish. We can provide own abstractions to our plugins, allowing us to provide developers with a more consistent API and eliminate boilerplate code necessary to extend the platform with new plugins.

In short, we want to move fast: it should be possible to prototype a new course layout within a couple of days.

Test everything

Course formats make or break a platform.

Since users spend such a large portion of their time working with the course interface, it's incredibly important that we get it right. Adding interactivity to a platform is great, but it's worthless if it provides an inconsistent user experience.

This is also an essential requirement for continued development. As we're building large numbers of plugins around a core framework, it's essential that we're able to verify that changes to that core aren't breaking any public API.

Technical implementation

Fluid is comprised of two core plugins:

  • format_fluid is the core of the platform.
  • local_fluid is merely a container for the subplugins. There's no logic in here, just the pluginfo classes for the three plugin types and accompanying language strings.

Fluid is designed to provide a framework, layered over Moodle's course formats, which allows developers to rapidly build additional user interfaces without having to consider the technical complexity of course formats.

Monkey patching core

To simplify maintenance of our platforms, Floream maintains a policy against patching Moodle core. This policy presented challenges, as Fluid frequently hit the limits of the course format API.

Ordinarily, I would not advise following practices like this in production grade code. Every major Moodle version or series presents a non-trivial risk of breakage to Fluid because we're interfering with API that can be considered private. We would like to address these problems in the long term by submitting patches upstream, but we believe our acceptance tests adequately offset this risk in the meantime.

A notable course format limitation is the inability of a format plugin to attach custom data to activities and resources: the course format API only provides API for extending the course and section edit forms via format_base::create_edit_form_elements().

To work around this, we strategically abuse format_base::has_view_page(), which is called from the moodleform_mod::add_action_buttons() method, allowing us to obtain the MoodleQuickForm object we need to add our elements to. We then use reflection to obtain the cm_info object so we know which CM we should be applying changes to.

public function has_view_page() {
    /* Monkey patch the module edit form, if possible. Since this method is
     * called from many places outside of course module edits, we have to
     * avoid throwing warnings when we're not called as part of this
     * process. */
    $backtrace = debug_backtrace();
    if (array_key_exists('class', $backtrace[1])
            && $backtrace[1]['class'] === 'moodleform_mod'
            && $backtrace[1]['function'] === 'add_action_buttons') {
        try {
            $modform = backtrace_helper::find_module_form();

            $reflector = new ReflectionProperty('moodleform_mod', '_cm');
            $reflector->setAccessible(true);
            $cm = $reflector->getValue($modform);
            $reflector->setAccessible(false);

            $cmid = ($cm === null) ? 0 : $cm->id;
            $this->hook_mod_form($modform, $cmid);
        } catch (backtrace_find_exception $e) {
            // Is this really useful? We might want to remove it now...
            debugging(static::DEBUG_MISSING_FORM);
        }
    }

    /* TODO: this decision should be made by the Fluid course layout, but
     *       we haven't implemented it yet. */
    return true;
}

We then use an event observer to subsequently save the values, but again have to seek out the form object:

public static function course_module_created(course_module_created $eventdata) {
    global $CFG;

    $format = course_get_format($eventdata->courseid);
    if (!$format instanceof format_fluid) {
        return;
    }

    try {
        $modform = backtrace_helper::find_module_form();
    } catch (backtrace_find_exception $e){
        debugging(format_fluid::DEBUG_MISSING_FORM);
        return;
    }

    $format->hook_mod_form($modform, $eventdata->objectid);
}

backtrace_helper exists to consolidate some particularly ugly and version-specific source. For example, the method below walks the call stack to locate the moodleform_mod object for the course module being edited, which is the context ($this) object for the moodleform_mod::add_action_buttons() method call:

public static function find_module_form() {
    $backtrace = debug_backtrace();

    foreach ($backtrace as $trace) {
        if ($trace['function'] === 'update_moduleinfo') {
            return $trace['args'][3];
        } elseif ($trace['function'] === 'add_moduleinfo') {
            return $trace['args'][2];
        } elseif (array_key_exists('class', $trace)
                && $trace['class'] === 'moodleform_mod'
                && $trace['function'] === 'add_action_buttons') {
            return $trace['object'];
        }
    }

    throw new backtrace_find_exception($backtrace);
}

Whilst techniques like these are necessary for Fluid to function in the short term, we do plan to collaborate with the Moodle community to develop and merge changes to the course format API that will enable us to remove these workarounds.

Working around questionable API design

First, a little background. All course format classes (including format_fluid), inherit from format_base. The constructor of this class expects a $courseid. This makes perfect sense, as the format is loaded to render an individual course.

The complexity arises when we're considering the creation of new courses. Because the course record doesn't yet exist, we don't have an ID.

/* Guard against stupid functions like course_format_uses_sections()
 * which don't provide us with a course ID or record and instead stub
 * out a stdClass instance with the format property. */
if ($courseid == 0 && $COURSE->id !== SITEID) {
    $courseid = $COURSE->id;
}

Testing all the things

In testing Fluid, we very quickly ran into some problems:

  • Moodle core's Behat step definitions were sometimes inadequate for our testing, using either overly specific selectors to locate elements (such as section-related steps assuming each section is a list item) or leaving an unreasonable degree of complexity to the author of the test (e.g. changing the Format field of a course on the Edit settings page doesn't trigger a refresh, requring multiple passes).

    To this end, we started collecting our most useful Behat step definitions and release them independently of Fluid in the Mad Behatter plugin.

  • The YUI BootstrapEngine, used in Moodle in lieu of an "official" port of the Bootstrap JS to YUI, is only a partial implementation of the Bootstrap JS. This was a source of much frustration, as it potentially required us to write our own implementations of the collapse and tab modules for a platform which has already been declared end of life.

    The simpler solutions usually win out: we opted to replace this code with the stock Bootstrap jQuery plugin within our themes. It worked. We run the Fluid acceptance tests against a child of the Clean theme which merely includes jQuery and the Bootstrap 2.3.x JS in its theme_*_page_init() function.

Opt out of rendering the editing UI

The Moodle course editing UI places a great deal of emphasis on drag and drop functionality, allowing users to reorder activities and resources and move them between sections. Unfortunately, the implementation of this functionality builds on the assumption that courses are a list of sections, each containing a list of activities and resources.

Most Moodle course formats either:

  • implement a separate code path for editing which outputs the course in list form; or
  • ship their own implementation of the editing JavaScript which works with their own markup.

Neither of these options are appealing to us, as they require us to duplicate effort across our different course and section layouts. The inherent complexity of the interactions between the two plugin types is also undesirable.

Our solution to this problem was to simply not attempt to implement editing support using Fluid's layouts, instead opting to pass through to the Topics course format in Moodle core. This has the disadvantage of not providing users with a live WYSIWYG preview of the content they're working with. In practice, this has not proven to be an issue internally, as users will preview courses in a different browser session logged in as a student.

How the renderers interact

When rendering a Moodle course, there are usually two key classes in use within the course format:

  • core_course_renderer renders the activities and resources within sections. It's generally overridden for all course formats within the theme as opposed to being modified within course formats.
  • format_section_renderer_base is the main extension point for Moodle course formats.

Fluid introduces subclasses of each of these, format_fluid_sectionlayout_renderer and format_fluid_courselayout_renderer respectively. The format_fluid_renderer class merely implements very simple editing functionality and the glue for rendering a course view with the different layout renderers.

We introduced some additional methods on the format class which facilitate easy location of the different layout components:

  • format_fluid::get_course_layout($name=null) returns the named layout class, or the course default if none was was specified.
  • format_fluid::get_course_renderer() returns the course layout configured for use with the course.
  • format_fluid::get_section_layout($name=null) returns the named layout class, or the course default if none was was specified.
  • format_fluid::get_section_renderer($section=null) returns the section layout renderer for the named section, or the course default if none was specified.

We initially attempted to implement all of the layout-related functionality within the renderer classes, but this proved problematic for some methods called during early initialisation. We opted to implement a subset of the format_base methods in our course_layout and section_layout classes instead.

Option inheritance

Fluid allows content creators to configure layout options on a per-section basis. Whilst this feature is powerful, it has the potential to introduce a non-trivial amount of administration overhead.

To alleviate this problem, Fluid implements inheritance of section-level options from the parent course. Where possible, Fluid will favour the values set at the section level, falling back to the course default where a value for an option cannot be found within the section.

Layout plugins

Course layouts:

  • Collapsed is similar to the existing Collapsed Topics format, representing each section as a collapsible region in an accordion-style layout. Our implementation can be configured to allow expanding only a single section at a time, and persists the user's expansion preferences for the lifetime of their session.
  • List is a faithful reimplementation of the existing topics course format, presenting all sections on the course view page.
  • Tabbed was the first course layout we developed and deployed. It provides a modern Bootstrap 3 take on the existing Tab Topics format.

Section layouts:

  • Carousel renders each activity as a tile in a carousel, usually accompanied by a thumbnail. The number of tiles visible at once is configurable and it's possible to have the section loop back to the beginning if the user attempts to scroll beyond the end.
  • Grid is similar to the carousel layout, except it's implemented as a Bootstrap grid. This allows for 1, 2, 3, 4 or 6 activities per row where labels can optionally break the grid, being displayed on their own dedicated rows.
  • List is another faithful reimplementation of the topics course format, presenting each activity in a list item.

Metadata plugins

  • The Activity type plugin simply provides a custom field for the theme to print as a CSS class on each activity's root element (list item or grid cell).
  • Guided learning hours allows a estimated duration (and units, for presentation purposes) to be specified for each activity.
  • Thumbnails are used to provide more appropriate images for presentation as part of a grid or carousel section layout.

Notable customisations

  • On the Home Learning College campus, we share a single trait for the course module tiles between both the carousel and grid layouts. This is one of the great strengths of adopting Moodle's renderer system wholeheartedly vs traditional templating: we're able to easily share code through inheritance.

Lessons learned

The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.

Tom Cargill, Bell Labs

A lot of finesse goes into replicating little-known features of a system as complex as the Moodle course interface. Two examples that immediately spring to mind:

  • Orphaned sections, created by setting the number of visible sections within the course (maxsections) to a number lower than the number of existing sections, are commonly used on our platforms.
  • Section zero (the "general" section) in Fluid is considered to be static, exempt from scheduling and always visible, rendered independently of any collapsible regions that might obscure it.

Future ideas

Fluid is very much a moving target, and we've got many ideas for improving the core framework:

  • We've not yet reviewed the accessibility of the current Fluid layout plugins. Whilst we're playing to the strengths of Bootstrap wherever possible, it pains me to admit that we haven't yet performed hands on testing of the platforms with screen readers.
  • Metadata plugins should be LT-configurable. We could amend Fluid to simply provide a set of field types, and allow learning technologists to configure their own fields at each level of the hierarchy.
  • There's certainly plenty of performance optimisation left to be done. We should start with the low hanging fruit on the server side, reducing the amount of work necessary to locate section layout plugins and their associated renderers. Reflection is expensive, and the debugging functions we're using aren't optimised for heavy use in production. In the long term, we could look at introducing dynamic loading of content with AJAX. I suspect larger courses (e.g. 52 weeks worth of sections!) using the tabbed and collapsible course layouts would benefit massively from this work.
  • Improve the documentation. The inline PHPDocs in the source often don't match the function prototypes or lack detail on certain parameters. We should also invest in signposting for developers not familiar with the Fluid internals.
  • Flexible activity availability restrictions. The core implementation of availability restrictions on activities is complex and limiting from a user experience perspective, merely providing developers with access to a string explaining the restrictions. To do better than this, we'll need to provide developers with finer grained access to the restriction data.
  • Configurable grid layouts. The grid section layout is a great start, but at the moment it provides content creators with no means to set the widths or positions of individual activities. I'm not sure we'll be able to address this in the near future, as it would require a major rethink of our approach to the editing interface.
  • We should provide additional scheduling options for sections. The current implementation simply dates the sections at weekly intervals from the course's start date.
    • "x units from previous section start date"?
    • Making the dates relative to the enrolment start date, allowing multiple cohorts of users to progress through a single course at different times?

Whilst we've covered our core requirements for Fluid, we have plenty of ideas for additional plugins:

  • A Masonry section layout for compact presentation of activities where sequencing isn't important. This would challenge the notion of content being sorted into a linear sequence, and likely wouldn't work for the majority of content.
  • A searching/filtering section layout, where activities are filtered against a user-entered set of search terms.
  • A custom LESS/SASS metadata plugin. Users would configure global templates of LESS/SASS/SCSS which make use of variables, then specify values for the variables within each course. As if by magic, Fluid would generate and cache the necessary CSS per-course.

Things we'd definitely do differently

  • Implement backup for the metadata plugins from the get go. This became a considerable time sink during our content deployments, as migration scripts would have to be manually authored by a developer as part of a release.
  • Place additional emphasis on the performance of the platform, particularly for larger courses comprised of a large number of sections.

*[WYSIWYG]: What You See Is What You Get