1/33

The Canvas learning management system
and LATEXMLThe LATEX workﬂow is still the best
Will Robertson

July 20, 2018

2/33

Introduction
What is a learning management system?
Speciﬁcations for the project
The authoring interface
Which ‘HTML’ tool to choose?
LATEXML overview
The Canvas programming interface
The generated html page
Images and ﬁlesUncertainties
Beneﬁts of scripting Canvas
Conclusion

Introduction

Introduction
What is a learning management system?
Speciﬁcations for the project
The authoring interface
Which ‘HTML’ tool to choose?
LATEXML overview
The Canvas programming interface
The generated html page
Images and ﬁlesUncertainties
Beneﬁts of scripting Canvas
Conclusion

3/33

Introduction

Background• The University of Adelaide recently switched to Canvas asits LMS• The LMS has an API!
• I wanted to sensibly manage ‘content’• Generate both HTML and PDF• A good excuse to learn LATEXML…

4/33

Introduction
What is a learning management system?

This is an LMS

5/33

Introduction
Speciﬁcations for the project

Motivation

1. In the old days, had a monolithic (impossible) Word ﬁle
2. Care and maintenance of content• Click click click• No good tools to manage content globally

3. Re-use of content for a comprehensive PDF• ‘One stop shop’ for reference• Handy to be able to distribute(for students, for supervisors, for accreditation)

6/33

Introduction
Speciﬁcations for the project• One source, multiple outputs• Macros to ensure up-to-date info
• Names of certain people• Due dates• Weightings of assessments• Reliable (easy setup, actively developed, etc.)• LATEX based (sorry, not sorry)

7/33

The authoring interface

Introduction
What is a learning management system?
Speciﬁcations for the project
The authoring interface
Which ‘HTML’ tool to choose?
LATEXML overview
The Canvas programming interface
The generated html page
Images and ﬁlesUncertainties
Beneﬁts of scripting Canvas
Conclusion

8/33

The authoring interface
Which ‘HTML’ tool to choose?

A number of LATEX to html possibilities, including:• TEX4ht — (?), in TEX Live• lwarp — Lua, in TEX Live (951 p. manual)• LATEXML — Perl, not in TEX Live• HEVEA — OCaml, not in TEX Live
(Non-exhaustive list. All are actively developed.)

9/33

The authoring interface
Which ‘HTML’ tool to choose?

• Hopefully the choice is not too important!
• I.e., the conversion aspect should be modular• I wanted to try LATEXML, and I’ve been happy (enough)with it to date.

10/33

The authoring interface
LATEXML overview

Architecture of LATEXML
Caveat: I’m only a user!
• Reimplements some TEX scanning in Perl
• Therefore handles basic LATEX 2𝜀 programming• \newcommand• \newcounter, \stepcounter• etc.

• Provides Perl interfaces to emulate classes and packages
A combination of LATEX 2𝜀 , Perl, XSLT, HTML, CSS, …
Anyway, the basics works well:\newcommand\honourscoord{Will Robertson}

11/33

The authoring interface
LATEXML overview

Running LATEXML

Two-phase process: latexml then latexmlpost
latexml tex/$FILENAME.tex | latexmlpost - \--xsltparameter=SIMPLIFY_HTML:true \--sourcedirectory=tex \--format=html5 \--destination=html/$FILENAME.html \--splitat=chapter \--splitnaming=label

12/33

The Canvas programming interface

Introduction
What is a learning management system?
Speciﬁcations for the project
The authoring interface
Which ‘HTML’ tool to choose?
LATEXML overview
The Canvas programming interface
The generated html page
Images and ﬁlesUncertainties
Beneﬁts of scripting Canvas
Conclusion

13/33

The Canvas programming interface

Canvas programming• A so-called rest api
Wikipedia: ‘REST-compliant web services allow the requesting systems toaccess and manipulate textual representations of web resources by using auniform and predeﬁned set of stateless operations.’• Using curl:
curl -X GET -H "$CANVASAUTH" $CANVASCOURSE/$1• $CANVASAUTH = secret token• $CANVASCOURSE = url to course• $1 = users or rubrics orassignments?search_term=charter etc.

• E.g.:
curl -X GET -H 'Authorization: Bearer 81...Jx'
https://myuni.adelaide.edu.au/api/v1/courses/36028/assignments

14/33

The Canvas programming interface

15/33

Uploading a ﬁle to Canvas
Clear as mudcurl -X POST -H "$CANVASAUTH" "$CANVASCOURSE/files"\-F "name=$1" -F "parent_folder_path=upload" > tmp.json ;URL=`cat tmp.json | jq '.upload_url'` ;KEYS=`cat tmp.json | jq '.upload_params' | jq -r -j \"to_entries | map(\"-F \(.key)=\(.value|tostring) \")|.[]"` ;echo curl -D response.tmp $URL $KEYS -F file=@$1 | bash ;LOC=`sed -n -e 's/Location: \(.*\)/\1/p' response.tmp`;
LOC=${LOC%$'\r'}
curl -X POST -H "$CANVASAUTH" "$LOC" | jq ;• I have forgotten how this works!
• Allows me to, say, upload the typeset PDF automaticallyafter updating content.

The Canvas programming interface

Evolution of my support scripts• Started with curl• A few small-ish Bash functions and scripts• Have now started with Lua programming• Requesting data with many items (‘all submittedassignments’, say) is returned in multiple ‘pages’ so
iteration is required• Much prefer doing real programming not in Bash
• Start looking into the Lua package ecosystem

16/33

The Canvas programming interface

Package proliferation: not just a LATEX problem• luajson by harningt• lunajson by graﬁ
• rapidjson by xpol
• dkjson by dhkolf• jwt-jitsi by pawelgawel88• JSON4Lua by luarocks• jwt by olivine-labs• mjolnir._asm.data.json by _asm
• ngxjsonform by rtbz
• dromozoa-json by moyu
• lua-cjson-ol by olivine-labs• lua-cjson2 by CriztianiX• json-lua by jiyinyiyong
• …

17/33

The Canvas programming interface

Lua equivalent to curl
local httplocal ltn12= require("ssl.https")= require("ltn12")

local body, code, headers, status = http.request{
method = "GET",
url = canvas_url .. req .. "?" .. opt,
headers = {["authorization"] = "Bearer " .. canvas_token,["content-type"] = "application/json"},sink = ltn12.sink.table(canvas_result),}• This returns json, which is ‘decoded’ into a Lua table.

18/33

The generated html page

Introduction
What is a learning management system?
Speciﬁcations for the project
The authoring interface
Which ‘HTML’ tool to choose?
LATEXML overview
The Canvas programming interface
The generated html page
Images and ﬁlesUncertainties
Beneﬁts of scripting Canvas
Conclusion

19/33

The generated html pageThe source\documentclass{report}\usepackage{latexml}...
\begin{document}\title{...}\author{...}\date{...}\maketitle\tableofcontents\input{../texdata/data-limits.tex}\input{../texdata/data-marks.tex}\part{Introduction}\label{part-intro}\input{../pages/introduction}\input{../pages/course-schedule}\input{../pages/week-planner}

20/33

The generated html page

• Each .tex ﬁle contains one chapter:\chapter{Introduction}\label{introduction}% same as filename!
...
• LATEXML does not convert ﬁle by ﬁle• Rather, output html is split by chapter• With consistent naming, this produces one .html ﬁle per.tex ﬁle• A ‘table of contents’ page is also generated

21/33

The generated html pageTop matter<!DOCTYPE html><html><head><title>XX Chapter title</title><meta http-equiv="Content-Type"content="text/html; charset=UTF-8"><link rel="stylesheet" href="LaTeXML.css"
type="text/css">[...]</head><body>

22/33

The generated html page

Bottom matter<footer class="ltx_page_footer">[...]</footer></div></body></html>23/33

The generated html page

‘Content’<div class="ltx_page_main"><header class="ltx_page_header">[...]</header><div class="ltx_page_content"><section class="ltx_chapter ltx_authors_1line"><h1 class="ltx_title ltx_title_chapter"><span class="ltx_tag ltx_tag_chapter">Chapter X </span>
Title of chapter goes here</h1>
<div class="ltx_date ltx_role_creation"></div><section id="S1" class="ltx_section">[...]
ALL THE CONTENT[...]</section></section></div>

24/33

The generated html page

25/33

‘Content’<section class="ltx_chapter ltx_authors_1line"><h1 class="ltx_title ltx_title_chapter"><span class="ltx_tag ltx_tag_chapter">Chapter X </spa
Title of chapter goes here</h1><div class="ltx_date ltx_role_creation"></div><section id="S1" class="ltx_section">[...]
ALL THE CONTENT[...]</section></section>

The generated html page

One line of awk• Snipping is done withawk '/\<section.*\>/,/\<\/section\>/' \html/$BASE >> snip/$BASE• Possible to take the raw xml and develop own workﬂows,
but unnecessary for me since I want html anyway.

26/33

Images and ﬁlesIntroduction
What is a learning management system?
Speciﬁcations for the project
The authoring interface
Which ‘HTML’ tool to choose?
LATEXML overview
The Canvas programming interface
The generated html page
Images and ﬁlesUncertainties
Beneﬁts of scripting Canvas
Conclusion

27/33

Images and ﬁlesNot much to say here:• Using sensible ﬁle structure, match up location of images
for the pdf on the disk, and for the html in the server• Same thing for ﬁles, but currently I don’t link ﬁles!
(TODO)

28/33

Uncertainties

Introduction
What is a learning management system?
Speciﬁcations for the project
The authoring interface
Which ‘HTML’ tool to choose?
LATEXML overview
The Canvas programming interface
The generated html page
Images and ﬁlesUncertainties
Beneﬁts of scripting Canvas
Conclusion

29/33

Uncertainties

Uncertainties• Maths? Not well supported in Canvas, yet. That’s okay —for this course I don’t need maths.
But this is really important!!
• Bash scripts? One step at a time.
• All Lua, eventually — cross-platform and a bit moresensible.
• How far do I go writing a general Canvas interface?

30/33

Uncertainties

Is LATEXML the right approach?

• The Perl layer is a little foreign to me but appears
well-designed.
• The xml output appears highly ﬂexible.
• In the long-run, what does LATEX itself need to provide?
• Could LATEXML be written in TEX itself? Or LuaTEX?

31/33

Beneﬁts of scripting Canvas

Live demo???

(Not sure if this is a good idea…)

32/33

Conclusion

Introduction
What is a learning management system?
Speciﬁcations for the project
The authoring interface
Which ‘HTML’ tool to choose?
LATEXML overview
The Canvas programming interface
The generated html page
Images and ﬁlesUncertainties
Beneﬁts of scripting Canvas
Conclusion

33/33

Conclusion

Summary• From LATEX (to pdf, and) to LATEXML to Canvas.
• LATEX is still the pre-eminent document preparation
system.
• LuaTEX opens the door for a tightly integrated approachfor bidirectional transfer of information between LATEXand Canvas.

34/33