Experimenting With Hypothesis

Table of Contents

Part 1: What are OLCCS?
Part 2: What is Hypothes.is (h.)
Part 3: Integration of Hypothes.is into an OLCC
Part 4: Thoughts and Reflections

Part 1:  What are OLCCs?

OLCCs are OnLine Chemistry Courses that have been run by the CCCE since 1996, please see the Spring 2016 ConfChem article on the Twentieth Anniversary of the OLCC[1] for more information  Today OLCCs are run on a Drupal site that is very similar to the one this newsletter article is run on. The fundamental difference between an OLCC Module and a ConfChem or Newsletter article is that the later can only be discussed at the bottom of each page, while the former is broken into multiple sections that can be discussed independent of each other.  This is done by dividing OLCC Modules into TLOs (Teaching and Learning Objects), with each TLO having its own discussion section.).  

1.1 OLCC Video Tutorials

To assist students in becoming familiar with the website we created a series of YouTube tutorials on how to use the site.  If you are not familiar with OLCCs you may want to watch a few of the following video tutorials.

Video 1.1a: Introduction to OLCCs
http://olcc.ccce.divched.org/Spring2017OLCCWebTutorialsTLO2TLOf (2:08 min)

This first video was not only designed to introduce students to the concept of an OLCC, but also make them feel they were part of something of historic significance. That is, the video goes through the Way Back Machine of the Internet Archive to look at the very first OLCC, which was probably run before some of these students were even born.

Video 1.1b: Logging in and Discussing Modules
http://olcc.ccce.divched.org/Spring2017OLCCWebTutorialsTLO1 (6:33 min)

This video shows how we discuss modules (before hypothes.is), which is the same way we discuss this newsletter, except that you could discuss an article at multiple sections.

Video 1.1c: OLCC Web Site Overview
http://olcc.ccce.divched.org/Spring2017OLCCWebTutorialsTLO2TLOe (8:36)

This video introduced the course website in the context of an eTextbook and described some of the Drupal features that the site was built upon. The idea was if students could understand how the site was constructed, they would be better able to use the web site, especially with respect of the ability of students to contribute TLOs (Teaching and Learning Objects) into the site.

1.2 Spring 2017 Cheminformatics OLCC

The 2017 Cheminformatics OLCC course was taught at 10 different schools and broken 4 Sections.

            Section 1: Cheminformatics and Data Representations
            Section 2: Public Compound Databases (Focus on PubChem)
            Section 3: Student Projects
            Section 4: Special Topics Modules

Sections 1 & 2 were the actual core modules that all students participated in, discussing the papers over the OLCCFAC and OLCCSTU lists. The first part dealt with ways chemicals were represented on computer and was taught during the month of February. Thanks to Bob Hanson, we had access to an interactive visualization, “Hack-a-mol,” [2], that allowed students to change mol files and see the result. The second part was taught in March and involved exercises and activities in PubChem. During the month of April students worked on Projects, that they presented to their individual classes.

The special topics modules spanned the entire semester, starting in January, with students and faculty self-subscribing to ones they were interested. Elsevier offered free access to Reaxys, a powerful chemical information portal to multiple proprietary and public compound databases, and several schools that started in January required students to take these modules while they were waiting for all schools Spring semesters to begin.

Running an OLCC is a huge challenge with the 2017 involving collaborative teaching and learning with active participants from across America, Germany, the Netherlands, Spain, England and Australia. One of the objectives behind the concept of an OLCC is through these collaborative interactions to allow an instructor to teach a topic they may not be an expert in, which is no easy undertaking. In fact, I would call that, the fact that the instructor can be a novice in the topic, to be the biggest challenge of offering an OLCC. And what I discovered, albeit not until the course was almost over, was that hypothes.is (h.) was a tool that could be used to great value in this type of dispersed expertise is integrated into collaborative teaching and learning.

Part 2: What is Hypothes.is  (h.) ? 

2.1 Introduction to Annotations

Hypothes.is (h.), https://hypothes.is , is an open source web annotation service run by a non-profit organization that was founded by Dan Whaley in 2011. The basic idea of web annotations, where a user selects text on a web page and then makes notes and discuss it with others in an overlay, has been around since the dawn of the web. In fact, it was even a feature of the early 0.4 version of NCSA Mosaic web browser. On February 23, 2017 the WC3 officially published its "Recommendations and Working Group Notes", or standards, that define how annotation clients and servers represent and exchange annotations, similar to the way that email clients and servers represent an exchange email messages. Those who are interested are encouraged to check this h. blog post, which includes a video with an interview of the likes of Tim Berners-Lee.

When an annotation refers to a phrase in a document. Hypothes.is provides a direct link, that combines the URL of the page with selectors that refer to the phrase within the page. This is exemplified in fig. 2.1a

2017CCENLP6fig7.JPG

Fig 2.1a: The target phrase (in blue) is a segment that is being annotated in the overlay

Hypothes.is actually provides two links, the first is to the webpage being annotated, which in the case of fig. 2.1a is  W3C Web Annotation Data Model , and the second is the enhanced or contextual link, which goes to the target segment of the annotation.

One other important feature of hypothes.is is that you can tag annotations or target segments of text. This allows a class to generate a folksonomy, which is a semantic web taxonomic information framework based on user-defined tags. Thus through the use of hypothes.is students can not only bring into the classroom discussion PDF files and web pages, but they also generate an information framework across disparate internet based resources that is germane to their assignments and learning objectives.  These discussions and social-tagged frameworks can be held in either the public space and be visible to the world, or done in one of the private Hypothes.is groups created for OLCC classroom specific collaborations.

2.2 Hypothes.is Education Resources

Hypothes.is has created a lot of tutorials for educators, and here are some links to their material for anyone who may be interested.

• Education Resources 
  • 10 Ways to Annotate with Students
  • Quick Start Guide for Teachers
  • Quick Start Guide for Students
  • Annotation Tips for Students
  • Using Hypothes.is Groups
• Hypothes.is YouTube Channel
• Resource Guides
  • Teacher Resource Guide
  • Student Resource Guide
• Hypothes.is Blog (has many blogs on classroom applications of h.)

Part 3: Integration of Hypothes.is into the Cheminformatics OLCC

3.1 Hypothes.is (h.) Annotations vs. Drupal Comment Based Discussions

When a class module was uploaded to the Drupal website it could be broken into multiple TLOs (Teaching and Learning Objects), each with its own comment feature, which effectively defined the information object being discussed (see video 1.1c). This allowed multiple concurrent discussions of an article, but the information architecture that defined the TLO comment-based discussions was static. With hypothes.is (h.), a dynamic information architecture for discussion could be spontaneously generated by any student or faculty member, who simply highlighted a section of text to start discussions of that section in the h. overlay.

Our earliest use of h. essentially emulated the way we used comments with the Drupal site, and we created h. groups for which an email would be triggered if an annotation was posted to that group. This not only allowed for the dynamic generation of the information object being discussed, but also allowed for objects outside of the module to discussed. Video 3.1a was created to introduce students of the Cheminformatics OLCC to h. and exemplified this early approach. For example, in the video a student used h. to trigger an email discussing an information object on the PubChem compound page for Acetaminophen, that explained why Acetaminophen showed up in a search for aspirin.
 

Video 3.1a:  http://olcc.ccce.divched.org/Spring2017OLCCWebTutorialsTLO3 (6:10 min). This video was created for the course website tutorial section and introduced students to hypothes.is and the two groups we created.


3.2 The Early Approach to the Use of Hypothes.is

The initial strategy was to set up two groups; OLCCNoMail and OLCCSendMail, both of which sent an RSS feed that the OLCC website collected and stored as a new Drupal content type, “OLCC annotation”. This allowed us to archive all of the hypothes.is annotations posted to those groups on our website, but we are still working on the best way to expose them to the public and integrate them with the Drupal comment-based discussions.  So the students had three options when posting an annotation, which are summarized in table 1.  Only annotations to the group “Public” were open access, and students had to belong to the groups (and be logged in) to see the other annotations. 

Table 3.2a: Summarizing the h. options students had available.

Through use of the two OLCC-class groups the initial strategy was to use h. to bring into the discussions online resources that were outside of the course website. OLCCSendMail essentially mimicked the Drupal comment-based discussions, in that an email with the annotation and a link back to the highlighted material was sent to everyone who was a member of the group (see embedded video 1 above). OLCCNoMail was to be used if we needed an annotation link for a Drupal comment, but did not want an email from to the h. group.

Unfortunately, there was a fatal flaw to implementing h. groups to the OLCC, in that we could not directly subscribe students to an h. group but had to send them a link to join, and so could not start out the class with everyone subscribed to the appropriate groups. Students also needed to learn how to use the Drupal comments at the beginning of the semester and this became too complicated to implement for a multi-campus course with semester start dates ranging from 1/9/2017 to 2/07/2017. Things were further exacerbated by the fact that we had no clear methodology for synchronizing the h. annotations with the Drupal comment-based discussions, and at least one instructor opted out of the h. annotations in favor of the time-tested Drupal comment-based discussions. The net result was that only public h. annotations were accessible to all students, and h. OLCC-group annotations could not be read by all students from all campuses, meaning the h. groups could not be used on the multi-institutional course level, but could be adapted on the single institution class level.

3.3 Initial observations of UALR student use of h. annotations concurrent with Drupal comments

UALR students were required to create h. accounts and since the other students could only see public comments, we ended up not using the groups, but posting the comments in public. The consequence was that UALR students became involved with two types communication activities, hypothes.is annotations and Drupal comments, and as the semester progressed it became clear that these different modes of discussion need not be redundant, but could be complementary.

When you think about it, the actual process of commenting and annotating are different. To make a comment, you effectively edit the page, in the sense that you post a title and comment that are now threaded into the discussion section of the page, and an email is sent to everyone subscribed. When you make an annotation, you do not bring up a form on the page, you simply highlight a section of text and click “annotate”, which causes an overlay on the side of the page to appear, which is like a piece of note paper. You make your comment on that overlay, not the page, and it is only observable if someone chooses to look at the annotations, by bringing up the overlay. That is, annotations mimic writing notes on the side of the page, while making comments adds to the content of the page itself.

The net result is that I noticed it was much easier for students to make annotations than to make comments. That is, students are used to making notes about papers, and the annotations modeled this established behavior. Making a comment was often difficult for students, who are novices, as that required them to add to the content of the paper itself, which was written by a domain expert.  In fact some students would not make comments, even if we developed their comment in class discussions and they were assigned points for posting comments.  But all students made annotations.

In summary, through hypothes.is they would read the papers and references and ask questions/make comments on the chapters and PDF files that were uploaded as supporting information, and the authors and I could go through them and get a bearing on where the students were at, and sometimes respond.  But if the students wanted a response from an author, they needed to use the comment feature of the Drupal website. 

3.4 Integrating hypothes.is into Drupal:

Around the 7^th week of the class we had a break-through and figured how to aggregate annotations on the topic of a webpage to the bottom of that webpage, as first implemented in module 2.3, (http://olcc.ccce.divched.org/2017OLCCModule2P3).  This was done by creating an h. tag based on the last part of the page's URL (the local address), and then setting up a Drupal token based on that address. So if a student tagged any web content with "2017OLCCModule2P3)", that annotation (and link) would be aggregated at the bottom of the above URL. This feature was integrated into Drupal content types OLCC Article and TLO (Teaching and Learning Object), and instructions were placed into the top of each page as shown in figure 3.4.a

2017CCENLP6fig1.png

Figure 3.4.a Top of a course webpage giving instructions on how to tag web content so it will be aggregated at the bottom of the page.

The h. aggregation box lists the title of each web page annotated and the number of annotations made to that page.  Fig 3.4.b shows that box for Module 2.3, which itself has 16 annotations, followed by a journal article with 1, and another with 3.  Note, the third link is really a PDF that was uploaded to the site as supporting information, and you need to be logged in to see it. The scroll bar allows you to go through all the material that has been tagged with the local URL of that webpage (2017OLCCModule2P3), and you can copy and paste it as one file.

2017CCENLP6fig2.png

Figure 3.4.b:  Hypothes.is (h.) aggregation box at the bottom of module 2.3, http://olcc.ccce.divched.org/2017OLCCModule2P3 .

In figure 3.4.c we see the material provided if you open one of the annotations in the h. annotation aggregator box (fig. 3.4.b). At the top is the source (Journal of Cheminformatics) and article (PubChem 3D: a new resource for chemists) the student is annotating. This is followed by the material the student highlighted in the article and the student's annotation.  Note, if the student highlighted an excessive amount of material an expandable teaser would appear.  If you click the "contextual link", the webpage will open in a new browser frame, scroll down to the annotated text, highlight it, and open the annotation in context (see Embedded Video 3.4) You can also copy and share both the link in context with the annotation (Copy Link URL), and the articles URL without the annotation.

2017CCENLP6fig3.png

Figure 3.4.c: Expanded View of an annotation within the aggregator.

Initially I expanded all the annotations and then copied and pasted them to a second document that I could edit and bring to class to guide me through the in-class discussions. Students were also replying to each other's annotations, as were the lecturers, although those discussions were not triggering emails to the other classes involved in the OLCC (they were still available, just as they are available to you, the reader of this article). I then realized that if I replied to the annotations I could filter the overlay to just show my replies as in figure 3.4.d.

2017CCENLP6fig4.png

Figure 3.4.d: Annotations in context, here, I have filtered the annotations by my replies, and if I click "View X more in conversation", where "X" is the number of other annotations, I can open-up all the discussion associated with this annotation, including the original student question I was replying. 

The above image exemplifies how you can use hypothes.is to prepare and give a lecture in a pseudo flipped classroom environment. In the annotation overlay I have filtered the annotations by the username “rebelford”.  The class was covering module 6, “How to Search PubChem for Chemical Information (Part 2)” (Structure Search), and the first annotation was one I made to clarify something in the article that I originally thought might be confusing.  The second annotation was a reply to a student question, and the third was a reply to a student question for which another student also joined the discussion (as there were 2 more comments in the discussion).  This strategy is outlined in the following YouTube, embedded video 1.4

Video 3.4e: https://youtu.be/-6vBO_dQCgc  (2:45 min). This video shows how one could use the ability to aggregate annotations at the bottom of a page to prepare for in-class lecture.

Students could now make annotations on the modules, any of the uploaded files (that required student login to access), or any open-access referenced article in the module, as well as online resources like the PubChem pages themselves, and these annotations would be aggregated together at the bottom of the module. By integrating tags with the annorations students were not only forced to analyze the references and various online resources this course covered, but also created a folksonomic framework on the topic of the module that integrated disparate resources across the web with the course module's content. The instructor could then use this linked folksonomy to assess student learning and identify areas of difficulty that would be good for in-class focus. Then by replying to chosen annotations the instructor could add to the folksonomy additional material targetting these needs, and then through a filter of the instructor's user-ID, reduce the content of the folksonomy to those items the instructor feels will be of greatest value to the class.

4. Thoughts and Reflections

The cheminformatics OLCC sought to enable faculty to teach a course focused on chemical data representation and public compound databases, when the faculty often had little to no formal training. This was done through a hybrid f2f/online format where online experts interacted with students and residential faculty facilitators, who met with the students in the classroom. These are very difficult courses to teach because the facilitator is often learning the material at the same time the students are. That is, part of the idea of an OLCC is to allow a facilitator to teach a class they normally could not teach, due to a lack of domain expertise, giving both the facilitator and the students the opportunity to learn new material.

During the Spring 2017 Cheminformatics OLCC the UALR students and faculty experimented with the hypothes.is annotation system and by the end of the semester had worked out a technique to support teaching and learning in the OLCC environment. Students were required to read the material beforehand and make a minimum of 6 annotations, of which only three could be on the actual modules themselves. The rest had to be on the references, or online resources like pages within PubChem, and each of these annotations had to be tagged by the local address on the URL of the module the students were working on. These resulted in the annotations being aggregated at the bottom of the page, forming a folksonomy across the course content and the web that was based upon the students learning experience.

Based on my experience this semester, here is my proposal for how I would use annotations and tags the next time to support learning in an OLCC hybrid environment. This approach would also be viable for literature research article type activities, where the article being discussed is posted on a Drupal page that can aggregate annotations.

Students would be required to make a minimum of 6 annotations for each module (research article), all tagged with the local URL of the module (Drupal page that hosts article), with at least 4 being on external resources.  These can be referenced citations from the article or online resources the students utilize in their studies.  These must be complete a full 24 hours before lecture or no credit is awarded. This generates a folksonomy that can be accessed from the module and provides a navigational pathway across disparate resources that is reflective of the module topic and student learning activities.

The facilitator then goes through these and when needed, researches the answers, seeking assistance from the online lecturers when appropriate. These are prioritized with the objective of generating a lesson plan through responses that are either replies to specific questions, or new instructor generated annotations that reflect the class questions and discussion in a manner that supports the instructor's goals for student learning.  This is done by tagging the responses with special “class/date tags,” which are used to filter through all the annotations during in-class activities and generate an in class navigational pathway based on the hypothes.is contextual links that directly connect to specific target segments of documents the students have been studying or the instructor wishes to cover.

Acknowledgements

I would like to acknowledge Jon Holmes of UW Madison and Jon Udell of hypothes.is for their time and assistance, for without their their willingness to work together and try new things, we never would have gotten this to work.