Sage's Doctesting Framework
Release 10.4

The Sage Development TeamJul 23, 2024

CONTENTS

1Classes involved in doctesting

1

2

Classes for sources of doctests

21

3

Processes for running doctests

39

4

Parsing docstrings

69

5

Reporting doctest results

99

6

Detecting external software

111

7

Test the doctesting framework

121

8

Utility functions

143

9

Fixtures to help testing functionality

149

10 Indices and Tables

157

Python Module Index

159

Index

161

i

ii

CHAPTERONE

CLASSES INVOLVED IN DOCTESTING

This module controls the various classes involved in doctesting.
AUTHORS:• David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
class sage.doctest.control.DocTestController(options, args)
Bases: SageObject
This class controls doctesting of files.
After creating it with appropriate options, call the run() method to run the doctests.
add_files()
Checks for the flags ‘–all’ and ‘–new’.
For each one present, this function adds the appropriate directories and files to the todo list.
EXAMPLES:
sage: from sage.doctest.control import (DocTestDefaults,....:
DocTestController)
sage: from sage.env import SAGE_SRC
sage: import tempfile
sage: with tempfile.NamedTemporaryFile() as f:....:
DD = DocTestDefaults(all=True, logfile=f.name)....:
DC = DocTestController(DD, [])....:DC.add_files()Doctesting ...
sage: os.path.join(SAGE_SRC, sage ) in DC.files
True>>> from sage.all import *>>> from sage.doctest.control import (DocTestDefaults,...
DocTestController)>>> from sage.env import SAGE_SRC>>> import tempfile>>> with tempfile.NamedTemporaryFile() as f:...
DD = DocTestDefaults(all=True, logfile=f.name)...
DC = DocTestController(DD, [])...
DC.add_files()Doctesting ...
>>> os.path.join(SAGE_SRC, sage ) in DC.files
True

1Sage's Doctesting Framework, Release 10.4

sage: DD = DocTestDefaults(new = True)
sage: DC = DocTestController(DD, [])sage: DC.add_files()Doctesting ...
>>> from sage.all import *>>> DD = DocTestDefaults(new = True)>>> DC = DocTestController(DD, [])>>> DC.add_files()Doctesting ...

cleanup(final=True)
Runs cleanup activities after actually running doctests.
In particular, saves the stats to disk and closes the logfile.
INPUT:• final – whether to close the logfile
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, sage , rings , all.py )sage: DD = DocTestDefaults()
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()sage: DC.sources.sort(key=lambda s:s.basename)sage: for i, source in enumerate(DC.sources):....:
DC.stats[source.basename] = { walltime : 0.1r * (i+1)}....:
sage: DC.run()
Running doctests with ID ...
Doctesting 1 file.
sage -t .../rings/all.py[... tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0sage: DC.cleanup()>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> from sage.env import SAGE_SRC>>> import os>>> dirname = os.path.join(SAGE_SRC, sage , rings , all.py )>>> DD = DocTestDefaults()(continues on next page)2

Chapter 1. Classes involved in doctestingSage's Doctesting Framework, Release 10.4

(continued from previous page)>>> DC = DocTestController(DD, [dirname])>>> DC.expand_files_into_sources()>>> DC.sources.sort(key=lambda s:s.basename)>>> for i, source in enumerate(DC.sources):...
DC.stats[source.basename] = { walltime : 0.1 * (i+Integer(1))}....:>>> DC.run()
Running doctests with ID ...
Doctesting 1 file.
sage -t .../rings/all.py[... tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0>>> DC.cleanup()

create_run_id()
Creates the run id.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DC = DocTestController(DocTestDefaults(), [])sage: DC.create_run_id()
Running doctests with ID ...
>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> DC = DocTestController(DocTestDefaults(), [])>>> DC.create_run_id()
Running doctests with ID ...

expand_files_into_sources()
Expands self.files,
FileDocTestSource

which may include directories,

into a list of sage.doctest.

This function also handles the optional command line option.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, sage , doctest )sage: DD = DocTestDefaults(optional= all )sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()sage: len(DC.sources)
12(continues on next page)3

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: DC.sources[0].options.optional
True>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> from sage.env import SAGE_SRC>>> import os>>> dirname = os.path.join(SAGE_SRC, sage , doctest )>>> DD = DocTestDefaults(optional= all )>>> DC = DocTestController(DD, [dirname])>>> DC.expand_files_into_sources()>>> len(DC.sources)
12>>> DC.sources[Integer(0)].options.optional
True
sage: DD = DocTestDefaults(optional= magma,guava )sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()sage: all(t in DC.sources[0].options.optional for t in [ magma , guava ])True>>> from sage.all import *>>> DD = DocTestDefaults(optional= magma,guava )>>> DC = DocTestController(DD, [dirname])>>> DC.expand_files_into_sources()>>> all(t in DC.sources[Integer(0)].options.optional for t in [ magma , guava
˓→ ])
True

We check that files are skipped appropriately:
sage: dirname = tmp_dir()sage: filename = os.path.join(dirname, not_tested.py )sage: with open(filename, w ) as f:....:_ = f.write("#"*80 + "\n\n\n\n## nodoctest\nsage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()sage: DC.sources[]

sage: 1+1\n>>> from sage.all import *>>> dirname = tmp_dir()>>> filename = os.path.join(dirname, not_tested.py )>>> with open(filename, w ) as f:...
_ = f.write("#"*Integer(80) + "\n\n\n\n## nodoctest\n˓→4")>>> DC = DocTestController(DD, [dirname])>>> DC.expand_files_into_sources()>>> DC.sources[]4")

sage: 1+1\n ␣

The directory sage/doctest/tests contains nodoctest.py but the files should still be tested whenthat directory is explicitly given (as opposed to being recursed into):

4

Chapter 1. Classes involved in doctestingSage's Doctesting Framework, Release 10.4

sage: DC = DocTestController(DD, [os.path.join(SAGE_SRC,
˓→ tests )])sage: DC.expand_files_into_sources()sage: len(DC.sources) >= 10
True>>> from sage.all import *>>> DC = DocTestController(DD, [os.path.join(SAGE_SRC,
˓→ tests )])>>> DC.expand_files_into_sources()>>> len(DC.sources) >= Integer(10)Truesage ,

sage ,

doctest ,doctest ,filter_sources()
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, sage , doctest )sage: DD = DocTestDefaults(failed=True)
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()sage: for i, source in enumerate(DC.sources):....:
DC.stats[source.basename] = { walltime : 0.1r * (i+1)}
sage: DC.stats[ sage.doctest.control ] = { failed : True, walltime : 1.0r}
sage: DC.filter_sources()
Only doctesting files that failed last test.
sage: len(DC.sources)
1>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> from sage.env import SAGE_SRC>>> import os>>> dirname = os.path.join(SAGE_SRC, sage , doctest )>>> DD = DocTestDefaults(failed=True)>>> DC = DocTestController(DD, [dirname])>>> DC.expand_files_into_sources()>>> for i, source in enumerate(DC.sources):...
DC.stats[source.basename] = { walltime : 0.1 * (i+Integer(1))}>>> DC.stats[ sage.doctest.control ] = { failed : True, walltime : 1.0}>>> DC.filter_sources()
Only doctesting files that failed last test.
>>> len(DC.sources)1

load_baseline_stats(filename)Load baseline stats.
This must be a JSON file in the same format that load_stats() expects.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DC = DocTestController(DocTestDefaults(), [])sage: import json(continues on next page)

5

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: filename = tmp_filename()
sage: with open(filename, w ) as stats_file:....:json.dump({ sage.doctest.control :{ failed :True}}, stats_file)
sage: DC.load_baseline_stats(filename)sage: DC.baseline_stats[ sage.doctest.control ]{ failed : True}>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> DC = DocTestController(DocTestDefaults(), [])>>> import json>>> filename = tmp_filename()>>> with open(filename, w ) as stats_file:...
json.dump({ sage.doctest.control :{ failed :True}}, stats_file)>>> DC.load_baseline_stats(filename)>>> DC.baseline_stats[ sage.doctest.control ]{ failed : True}

If the file doesn’t exist, nothing happens. If there is an error, print a message. In any case, leave the stats
alone:
sage: d = tmp_dir()sage: DC.load_baseline_stats(os.path.join(d)) # Cannot read a directory
Error loading baseline stats from ...
sage: DC.load_baseline_stats(os.path.join(d, "no_such_file"))sage: DC.baseline_stats[ sage.doctest.control ]{ failed : True}>>> from sage.all import *>>> d = tmp_dir()>>> DC.load_baseline_stats(os.path.join(d)) # Cannot read a directory
Error loading baseline stats from ...
>>> DC.load_baseline_stats(os.path.join(d, "no_such_file"))>>> DC.baseline_stats[ sage.doctest.control ]{ failed : True}

load_environment()
Return the module that provides the global environment.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DC = DocTestController(DocTestDefaults(), [])sage: BipartiteGraph in DC.load_environment().__dict__True
sage: DC = DocTestController(DocTestDefaults(environment= sage.doctest.all ),␣˓→[])
sage: BipartiteGraph in DC.load_environment().__dict__
False
sage: run_doctests in DC.load_environment().__dict__
True>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> DC = DocTestController(DocTestDefaults(), [])(continues on next page)

6

Chapter 1. Classes involved in doctestingSage's Doctesting Framework, Release 10.4

(continued from previous page)>>> BipartiteGraph in DC.load_environment().__dict__True>>> DC = DocTestController(DocTestDefaults(environment= sage.doctest.all ),␣˓→[])>>> BipartiteGraph in DC.load_environment().__dict__
False>>> run_doctests in DC.load_environment().__dict__Trueload_stats(filename)Load stats from the most recent run(s).
Stats are stored as a JSON file, and include information on which files failed tests and the walltime used forexecution of the doctests.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DC = DocTestController(DocTestDefaults(), [])sage: import json
sage: filename = tmp_filename()
sage: with open(filename, w ) as stats_file:....:json.dump({ sage.doctest.control : { walltime : 1.0r}}, stats_file)
sage: DC.load_stats(filename)sage: DC.stats[ sage.doctest.control ]{ walltime : 1.0}>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> DC = DocTestController(DocTestDefaults(), [])>>> import json>>> filename = tmp_filename()>>> with open(filename, w ) as stats_file:...
json.dump({ sage.doctest.control : { walltime : 1.0}}, stats_file)>>> DC.load_stats(filename)>>> DC.stats[ sage.doctest.control ]{ walltime : 1.0}

If the file doesn’t exist, nothing happens. If there is an error, print a message. In any case, leave the stats
alone:
sage: d = tmp_dir()sage: DC.load_stats(os.path.join(d)) # Cannot read a directory
Error loading stats from ...
sage: DC.load_stats(os.path.join(d, "no_such_file"))sage: DC.stats[ sage.doctest.control ]{ walltime : 1.0}>>> from sage.all import *>>> d = tmp_dir()>>> DC.load_stats(os.path.join(d)) # Cannot read a directory
Error loading stats from ...
>>> DC.load_stats(os.path.join(d, "no_such_file"))>>> DC.stats[ sage.doctest.control ]{ walltime : 1.0}

7Sage's Doctesting Framework, Release 10.4

log(s, end='\n')Log the string s + end (where end is a newline by default) to the logfile and print it to the standard output.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DD = DocTestDefaults(logfile=tmp_filename())
sage: DC = DocTestController(DD, [])sage: DC.log("hello world")
hello world
sage: DC.logfile.close()sage: with open(DD.logfile) as f:....:print(f.read())
hello world>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> DD = DocTestDefaults(logfile=tmp_filename())>>> DC = DocTestController(DD, [])>>> DC.log("hello world")
hello world>>> DC.logfile.close()>>> with open(DD.logfile) as f:...
print(f.read())
hello world

In serial mode, check that logging works even if stdout is redirected:
sage: DD = DocTestDefaults(logfile=tmp_filename(), serial=True)
sage: DC = DocTestController(DD, [])sage: from sage.doctest.forker import SageSpoofInOut
sage: with open(os.devnull, w ) as devnull:....:
S = SageSpoofInOut(devnull)....:S.start_spoofing()....:
DC.log("hello world")....:
S.stop_spoofing()
hello world
sage: DC.logfile.close()sage: with open(DD.logfile) as f:....:print(f.read())
hello world>>> from sage.all import *>>> DD = DocTestDefaults(logfile=tmp_filename(), serial=True)>>> DC = DocTestController(DD, [])>>> from sage.doctest.forker import SageSpoofInOut>>> with open(os.devnull, w ) as devnull:...
S = SageSpoofInOut(devnull)...
S.start_spoofing()...
DC.log("hello world")...
S.stop_spoofing()
hello world>>> DC.logfile.close()>>> with open(DD.logfile) as f:...
print(f.read())
hello world

Check that no duplicate logs appear, even when forking (Issue #15244):
8

Chapter 1. Classes involved in doctestingSage's Doctesting Framework, Release 10.4

sage: DD = DocTestDefaults(logfile=tmp_filename())
sage: DC = DocTestController(DD, [])sage: DC.log("hello world")
hello world
sage: if os.fork() == 0:....:
DC.logfile.close()....:
os._exit(0)sage: DC.logfile.close()sage: with open(DD.logfile) as f:....:print(f.read())
hello world>>> from sage.all import *>>> DD = DocTestDefaults(logfile=tmp_filename())>>> DC = DocTestController(DD, [])>>> DC.log("hello world")
hello world>>> if os.fork() == Integer(0):...
DC.logfile.close()...
os._exit(Integer(0))>>> DC.logfile.close()>>> with open(DD.logfile) as f:...
print(f.read())
hello worldrun()This function is called after initialization to set up and run all doctests.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: DD = DocTestDefaults()
sage: filename = os.path.join(SAGE_SRC, "sage", "sets", "non_negative_
˓→integers.py")
sage: DC = DocTestController(DD, [filename])
sage: DC.run()
Running doctests with ID ...
Doctesting 1 file.
sage -t .../sage/sets/non_negative_integers.py[... tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> from sage.env import SAGE_SRC>>> import os>>> DD = DocTestDefaults()>>> filename = os.path.join(SAGE_SRC, "sage", "sets", "non_negative_integers.
(continues on next page)9

Sage's Doctesting Framework, Release 10.4

(continued from previous page)py")>>> DC = DocTestController(DD, [filename])>>> DC.run()
Running doctests with ID ...
Doctesting 1 file.
sage -t .../sage/sets/non_negative_integers.py[... tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0˓→

We check that Issue #25378 is fixed (testing external packages while providing a logfile does not raise a
ValueError: I/O operation on closed file):
sage: logfile = tmp_filename(ext= .log )sage: DD = DocTestDefaults(optional=set([ sage , external ]),␣
˓→logfile=logfile)
sage: filename = tmp_filename(ext= .py )sage: DC = DocTestController(DD, [filename])
sage: DC.run()
Running doctests with ID ...
Using --optional=external,sageFeatures to be detected: ...
Doctesting 1 file.
sage -t ....py[0 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0>>> from sage.all import *>>> logfile = tmp_filename(ext= .log )>>> DD = DocTestDefaults(optional=set([ sage , external ]), logfile=logfile)>>> filename = tmp_filename(ext= .py )>>> DC = DocTestController(DD, [filename])>>> DC.run()
Running doctests with ID ...
Using --optional=external,sageFeatures to be detected: ...
Doctesting 1 file.
sage -t ....py[0 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds(continues on next page)10

Chapter 1. Classes involved in doctestingSage's Doctesting Framework, Release 10.4

(continued from previous page)

cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0

We test the --hide option (Issue #34185):
sage: from sage.doctest.control import test_hidesage: filename = tmp_filename(ext= .py )sage: with open(filename, w ) as f:....:f.write(test_hide)....:f.close()714
sage: DF = DocTestDefaults(hide= buckygen,all )sage: DC = DocTestController(DF, [filename])
sage: DC.run()
Running doctests with ID ...
Using --optional=sage...
Features to be detected: ...
Doctesting 1 file.
sage -t ....py[4 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0sage: DF = DocTestDefaults(hide= benzene,optional )sage: DC = DocTestController(DF, [filename])
sage: DC.run()
Running doctests with ID ...
Using --optional=sage
Features to be detected: ...
Doctesting 1 file.
sage -t ....py[4 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0>>> from sage.all import *>>> from sage.doctest.control import test_hide>>> filename = tmp_filename(ext= .py )>>> with open(filename, w ) as f:...
f.write(test_hide)...
f.close()714(continues on next page)

11

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> DF = DocTestDefaults(hide= buckygen,all )>>> DC = DocTestController(DF, [filename])>>> DC.run()
Running doctests with ID ...
Using --optional=sage...
Features to be detected: ...
Doctesting 1 file.
sage -t ....py[4 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0>>> DF = DocTestDefaults(hide= benzene,optional )>>> DC = DocTestController(DF, [filename])>>> DC.run()
Running doctests with ID ...
Using --optional=sage
Features to be detected: ...
Doctesting 1 file.
sage -t ....py[4 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
0

Test Features that have been hidden message:
sage: DC.run()# optional - meataxe
Running doctests with ID ...
Using --optional=sage
Features to be detected: ...
Doctesting 1 file.
sage -t ....py[4 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
Features that have been hidden: ...meataxe...
0>>> from sage.all import *(continues on next page)

12

Chapter 1. Classes involved in doctestingSage's Doctesting Framework, Release 10.4

(continued from previous page)>>> DC.run()# optional - meataxe
Running doctests with ID ...
Using --optional=sage
Features to be detected: ...
Doctesting 1 file.
sage -t ....py[4 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
Features that have been hidden: ...meataxe...
0

run_doctests()
Actually runs the doctests.
This function is called by run().
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, sage , rings , homset.py )sage: DD = DocTestDefaults()
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()sage: DC.run_doctests()
Doctesting 1 file.
sage -t .../sage/rings/homset.py[... tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds...
>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> from sage.env import SAGE_SRC>>> import os>>> dirname = os.path.join(SAGE_SRC, sage , rings , homset.py )>>> DD = DocTestDefaults()>>> DC = DocTestController(DD, [dirname])>>> DC.expand_files_into_sources()>>> DC.run_doctests()
Doctesting 1 file.
sage -t .../sage/rings/homset.py[... tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------(continues on next page)

13

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds...

run_val_gdb(testing=False)
Spawns a subprocess to run tests under the control of gdb, lldb, or valgrind.
INPUT:• testing – boolean; if True then the command to be run will be printed rather than a subprocess
started.
EXAMPLES:
Note that the command lines include unexpanded environment variables. It is safer to let the shell expand
them than to expand them here and risk insufficient quoting.
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DD = DocTestDefaults(gdb=True)
sage: DC = DocTestController(DD, ["hello_world.py"])sage: DC.run_val_gdb(testing=True)
exec gdb --eval-command="run" --args ...python... sage-runtests --serial -˓→timeout=0 hello_world.py>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> DD = DocTestDefaults(gdb=True)>>> DC = DocTestController(DD, ["hello_world.py"])>>> DC.run_val_gdb(testing=True)
exec gdb --eval-command="run" --args ...python... sage-runtests --serial -˓→timeout=0 hello_world.py
sage: DD = DocTestDefaults(valgrind=True, optional="all", timeout=172800)
sage: DC = DocTestController(DD, ["hello_world.py"])sage: DC.run_val_gdb(testing=True)
exec valgrind --tool=memcheck --leak-resolution=high --leak-check=full --num˓→callers=25 --suppressions="...valgrind/pyalloc.supp" --suppressions="...
˓→valgrind/sage.supp" --suppressions="...valgrind/sage-additional.supp"-˓→log-file=.../valgrind/sage-memcheck.%p... sage-runtests --serial -˓→timeout=172800 --optional=all hello_world.py>>> from sage.all import *>>> DD = DocTestDefaults(valgrind=True, optional="all",␣
˓→timeout=Integer(172800))>>> DC = DocTestController(DD, ["hello_world.py"])>>> DC.run_val_gdb(testing=True)
exec valgrind --tool=memcheck --leak-resolution=high --leak-check=full --num˓→callers=25 --suppressions="...valgrind/pyalloc.supp" --suppressions="...
˓→valgrind/sage.supp" --suppressions="...valgrind/sage-additional.supp"-˓→log-file=.../valgrind/sage-memcheck.%p... sage-runtests --serial -˓→timeout=172800 --optional=all hello_world.py

save_stats(filename)
Save stats from the most recent run as a JSON file.
WARNING: This function overwrites the file.
EXAMPLES:
14

Chapter 1. Classes involved in doctestingSage's Doctesting Framework, Release 10.4

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DC = DocTestController(DocTestDefaults(), [])sage: DC.stats[ sage.doctest.control ] = { walltime : 1.0r}
sage: filename = tmp_filename()
sage: DC.save_stats(filename)sage: import json
sage: with open(filename) as f:....:
D = json.load(f)sage: D[ sage.doctest.control ]{ walltime : 1.0}>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> DC = DocTestController(DocTestDefaults(), [])>>> DC.stats[ sage.doctest.control ] = { walltime : 1.0}>>> filename = tmp_filename()>>> DC.save_stats(filename)>>> import json>>> with open(filename) as f:...
D = json.load(f)>>> D[ sage.doctest.control ]{ walltime : 1.0}

second_on_modern_computer()
Return the wall time equivalent of a second on a modern computer.
OUTPUT:
Float. The wall time on your computer that would be equivalent to one second on a modern computer. Unless
you have kick-ass hardware this should always be >= 1.0. This raises a RuntimeError if there are no
stored timings to use as benchmark.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DC = DocTestController(DocTestDefaults(), [])sage: DC.second_on_modern_computer()# not tested>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> DC = DocTestController(DocTestDefaults(), [])>>> DC.second_on_modern_computer()# not tested

sort_sources()
This function sorts the sources so that slower doctests are run first.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, sage , doctest )sage: DD = DocTestDefaults(nthreads=2)
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()sage: DC.sources.sort(key=lambda s:s.basename)sage: for i, source in enumerate(DC.sources):(continues on next page)

15

Sage's Doctesting Framework, Release 10.4

(continued from previous page)....:DC.stats[source.basename] = { walltime : 0.1r * (i+1)}
sage: DC.sort_sources()
Sorting sources by runtime so that slower doctests are run first....
sage: print("\n".join(source.basename for source in DC.sources))sage.doctest.util
sage.doctest.test
sage.doctest.sources
sage.doctest.reporting
sage.doctest.parsing_test
sage.doctest.parsing
sage.doctest.forker
sage.doctest.fixtures
sage.doctest.external
sage.doctest.control
sage.doctest.all
sage.doctest>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> from sage.env import SAGE_SRC>>> import os>>> dirname = os.path.join(SAGE_SRC, sage , doctest )>>> DD = DocTestDefaults(nthreads=Integer(2))>>> DC = DocTestController(DD, [dirname])>>> DC.expand_files_into_sources()>>> DC.sources.sort(key=lambda s:s.basename)>>> for i, source in enumerate(DC.sources):...
DC.stats[source.basename] = { walltime : 0.1 * (i+Integer(1))}>>> DC.sort_sources()
Sorting sources by runtime so that slower doctests are run first....
>>> print("\n".join(source.basename for source in DC.sources))sage.doctest.util
sage.doctest.test
sage.doctest.sources
sage.doctest.reporting
sage.doctest.parsing_test
sage.doctest.parsing
sage.doctest.forker
sage.doctest.fixtures
sage.doctest.external
sage.doctest.control
sage.doctest.all
sage.doctest

source_baseline(source)
Return the baseline_stats value of source.
INPUT:• source – a DocTestSource instance
OUTPUT:
A dictionary.
EXAMPLES:

16

Chapter 1. Classes involved in doctestingSage's Doctesting Framework, Release 10.4

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: filename = sage.doctest.util.__file__
sage: DD = DocTestDefaults()
sage: DC = DocTestController(DD, [filename])
sage: DC.expand_files_into_sources()sage: DC.source_baseline(DC.sources[0]){}>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> filename = sage.doctest.util.__file__>>> DD = DocTestDefaults()>>> DC = DocTestController(DD, [filename])>>> DC.expand_files_into_sources()>>> DC.source_baseline(DC.sources[Integer(0)]){}

class sage.doctest.control.DocTestDefaults(**kwds)
Bases: SageObject
This class is used for doctesting the Sage doctest module.
It fills in attributes to be the same as the defaults defined in sage-runtests, expect for a few places, which ismostly to make doctesting more predictable.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: D = DocTestDefaults()
sage: D
DocTestDefaults()
sage: D.timeout-1>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> D = DocTestDefaults()>>> D
DocTestDefaults()>>> D.timeout-1

Keyword arguments become attributes:
sage: D = DocTestDefaults(timeout=100)
sage: D
DocTestDefaults(timeout=100)
sage: D.timeout
100>>> from sage.all import *>>> D = DocTestDefaults(timeout=Integer(100))>>> D
DocTestDefaults(timeout=100)>>> D.timeout
100

17Sage's Doctesting Framework, Release 10.4class sage.doctest.control.Logger(*files)
Bases: object
File-like object which implements writing to multiple files at once.
EXAMPLES:
sage: from sage.doctest.control import Logger
sage: with open(tmp_filename(), "w+") as t:....:
L = Logger(sys.stdout, t)....:_ = L.write("hello world\n")....:_ = t.seek(0)....:t.read()hello world
hello world\n>>> from sage.all import *>>> from sage.doctest.control import Logger>>> with open(tmp_filename(), "w+") as t:...
L = Logger(sys.stdout, t)...
_ = L.write("hello world\n")...
_ = t.seek(Integer(0))...
t.read()hello world
hello world\nflush()
Flush all files.
write(x)Write x to all files.
sage.doctest.control.run_doctests(module, options=None)
Runs the doctests in a given file.
INPUT:• module – a Sage module, a string, or a list of such.
• options – a DocTestDefaults object or None.
EXAMPLES:
sage: run_doctests(sage.rings.all)
Running doctests with ID ...
Doctesting 1 file.
sage -t .../sage/rings/all.py[... tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...
>>> from sage.all import *>>> run_doctests(sage.rings.all)
Running doctests with ID ...
(continues on next page)

18

Chapter 1. Classes involved in doctestingSage's Doctesting Framework, Release 10.4

(continued from previous page)

Doctesting 1 file.
sage -t .../sage/rings/all.py[... tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------Total time for all tests: ... seconds
cpu time: ... seconds
cumulative wall time: ... seconds
Features detected...

sage.doctest.control.skipdir(dirname)Return True if and only if the directory dirname should not be doctested.
EXAMPLES:
sage: from sage.doctest.control import skipdir
sage: skipdir(sage.env.SAGE_SRC)
False
sage: skipdir(os.path.join(sage.env.SAGE_SRC, "sage", "doctest", "tests"))True>>> from sage.all import *>>> from sage.doctest.control import skipdir>>> skipdir(sage.env.SAGE_SRC)
False>>> skipdir(os.path.join(sage.env.SAGE_SRC, "sage", "doctest", "tests"))Truesage.doctest.control.skipfile(filename, tested_optional_tags, if_installed, log=False)
Return True if and only if the file filename should not be doctested.
INPUT:• filename – name of a file• tested_optional_tags – a list or tuple or set of optional tags to test, or False (no optional test) or
True (all optional tests)• if_installed – (boolean, default False) whether to skip Python/Cython files that are not installed as
modules• log – function to call with log messages, or None
If filename contains a line of the form "# sage.doctest: optional - xyz"), then this will return
False if “xyz” is in tested_optional_tags. Otherwise, it returns the matching tag (“optional - xyz”).
EXAMPLES:
sage: from sage.doctest.control import skipfile
sage: skipfile("skipme.c")Truesage: filename = tmp_filename(ext=".pyx")
sage: skipfile(filename)False
sage: with open(filename, "w") as f:....:_ = f.write("# nodoctest")
sage: skipfile(filename)(continues on next page)

19

Sage's Doctesting Framework, Release 10.4

(continued from previous page)True
sage: with open(filename, "w") as f:....:_ = f.write("# sage.doctest: "˓→the pattern....:"optional - xyz")
˓→comment)
sage: skipfile(filename, False)
optional - xyz
sage: bool(skipfile(filename, False))
True
sage: skipfile(filename, [ abc ])optional - xyz
sage: skipfile(filename, [ abc , xyz ])False
sage: skipfile(filename, True)
False# broken in two source lines to avoid␣# of relint (multiline_doctest_>>> from sage.all import *>>> from sage.doctest.control import skipfile>>> skipfile("skipme.c")True>>> filename = tmp_filename(ext=".pyx")>>> skipfile(filename)
False>>> with open(filename, "w") as f:...
_ = f.write("# nodoctest")>>> skipfile(filename)True>>> with open(filename, "w") as f:...
_ = f.write("# sage.doctest: "# broken in two source lines to avoid␣˓→the pattern...
"optional - xyz")# of relint (multiline_doctest_comment)>>> skipfile(filename, False)
optional - xyz>>> bool(skipfile(filename, False))
True>>> skipfile(filename, [ abc ])optional - xyz>>> skipfile(filename, [ abc , xyz ])False>>> skipfile(filename, True)
False

20

Chapter 1. Classes involved in doctestingCHAPTERTWO

CLASSES FOR SOURCES OF DOCTESTSThis module defines various classes for sources from which doctests originate, such as files, functions or database entries.
AUTHORS:• David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
class sage.doctest.sources.DictAsObject(attrs)
Bases: dict
A simple subclass of dict that inserts the items from the initializing dictionary into attributes.
EXAMPLES:
sage: from sage.doctest.sources import DictAsObject
sage: D = DictAsObject({ a :2})
sage: D.a2>>> from sage.all import *>>> from sage.doctest.sources import DictAsObject>>> D = DictAsObject({ a :Integer(2)})>>> D.a2class sage.doctest.sources.DocTestSource(options)
Bases: object
This class provides a common base class for different sources of doctests.
INPUT:• options – a sage.doctest.control.DocTestDefaults instance or equivalent.
file_optional_tags()
Return the set of tags that should apply to all doctests in this source.
This default implementation just returns the empty set.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import StringDocTestSource, PythonSource
sage: from sage.structure.dynamic_class import dynamic_classsage: s = "\nsage: 2 + 2\n4\n"sage: PythonStringSource = dynamic_class( PythonStringSource ,␣˓→(StringDocTestSource, PythonSource))(continues on next page)

21

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: PSS = PythonStringSource( <runtime> , s, DocTestDefaults(),
sage: PSS.file_optional_tags
set()runtime )>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import StringDocTestSource, PythonSource>>> from sage.structure.dynamic_class import dynamic_class>>> s = "\nsage: 2 + 2\n4\n">>> PythonStringSource = dynamic_class( PythonStringSource ,␣˓→(StringDocTestSource, PythonSource))>>> PSS = PythonStringSource( <runtime> , s, DocTestDefaults(), runtime )>>> PSS.file_optional_tags
set()class sage.doctest.sources.FileDocTestSource(path, options)
Bases: DocTestSourceThis class creates doctests from a file.
INPUT:• path – string, the filename• options – a sage.doctest.control.DocTestDefaults instance or equivalent.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = sage.doctest.sources.__file__
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS.basename
sage.doctest.sources>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = sage.doctest.sources.__file__>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS.basename
sage.doctest.sources

basename()
The basename of this file source, e.g. sage.doctest.sources
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = os.path.join(SAGE_SRC, sage , rings , integer.pyx )sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS.basename
sage.rings.integer>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults(continues on next page)

22

Chapter 2. Classes for sources of doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> from sage.doctest.sources import FileDocTestSource>>> filename = os.path.join(SAGE_SRC, sage , rings , integer.pyx )>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS.basename
sage.rings.integer

create_doctests(namespace)
Return a list of doctests for this file.
INPUT:• namespace – a dictionary or sage.doctest.util.RecordingDict.
OUTPUT:• doctests – a list of doctests defined in this file.
• extras – a dictionary
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = sage.doctest.sources.__file__
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: doctests, extras = FDS.create_doctests(globals())
sage: len(doctests)
43
sage: extras[ tab ]False>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = sage.doctest.sources.__file__>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> doctests, extras = FDS.create_doctests(globals())>>> len(doctests)
43>>> extras[ tab ]False

We give a self referential example:
sage: doctests[20].namesage.doctest.sources.FileDocTestSource.create_doctests
sage: doctests[20].examples[8].source
doctests[Integer(20)].examples[Integer(8)].source\n>>> from sage.all import *>>> doctests[Integer(20)].namesage.doctest.sources.FileDocTestSource.create_doctests>>> doctests[Integer(20)].examples[Integer(8)].source
doctests[Integer(20)].examples[Integer(8)].source\nfile_optional_tags()
Return the set of tags that should apply to all doctests in this source.
EXAMPLES:
23

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = sage.repl.user_globals.__file__
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS.file_optional_tags{ sage.modules : None}>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = sage.repl.user_globals.__file__>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS.file_optional_tags{ sage.modules : None}in_lib()Whether this file is to be treated as a module in a Python package.
Such files aren’t loaded before running tests.
This uses is_package_or_sage_namespace_package_dir() but can be overridden via
DocTestDefaults.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: import os
sage: filename = os.path.join(SAGE_SRC, sage , rings , integer.pyx )sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS.in_lib
True
sage: filename = os.path.join(SAGE_SRC, sage , doctest , tests , abort.rst
˓→ )sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS.in_lib
False>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> import os>>> filename = os.path.join(SAGE_SRC, sage , rings , integer.pyx )>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS.in_lib
True>>> filename = os.path.join(SAGE_SRC, sage , doctest , tests , abort.rst )>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS.in_lib
False

You can override the default:
sage: FDS = FileDocTestSource("hello_world.py", DocTestDefaults())
sage: FDS.in_lib
False
sage: FDS = FileDocTestSource("hello_world.py", DocTestDefaults(force_(continues on next page)

24

Chapter 2. Classes for sources of doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

lib=True))sage: FDS.in_lib
True˓→>>> from sage.all import *>>> FDS = FileDocTestSource("hello_world.py", DocTestDefaults())>>> FDS.in_lib
False>>> FDS = FileDocTestSource("hello_world.py", DocTestDefaults(force_lib=True))>>> FDS.in_lib
True

printpath()
Whether the path is printed absolutely or relatively depends on an option.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: import os
sage: filename = os.path.realpath(sage.doctest.sources.__file__)sage: root = os.path.join(os.path.dirname(filename), .. )sage: cwd = os.getcwd()
sage: os.chdir(root)
sage: FDS = FileDocTestSource(filename, DocTestDefaults(randorder=0,....:
abspath=False))
sage: FDS.printpath
doctest/sources.py
sage: FDS = FileDocTestSource(filename, DocTestDefaults(randorder=0,....:
abspath=True))sage: FDS.printpath.../sage/doctest/sources.py
sage: os.chdir(cwd)>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> import os>>> filename = os.path.realpath(sage.doctest.sources.__file__)>>> root = os.path.join(os.path.dirname(filename), .. )>>> cwd = os.getcwd()>>> os.chdir(root)>>> FDS = FileDocTestSource(filename, DocTestDefaults(randorder=Integer(0),...
abspath=False))>>> FDS.printpath
doctest/sources.py>>> FDS = FileDocTestSource(filename, DocTestDefaults(randorder=Integer(0),...
abspath=True))>>> FDS.printpath.../sage/doctest/sources.py>>> os.chdir(cwd)class sage.doctest.sources.PythonSource
Bases: SourceLanguage
This class defines the functions needed for the extraction of doctests from python sources.

25

Sage's Doctesting Framework, Release 10.4

EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = sage.doctest.sources.__file__
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: type(FDS)<class sage.doctest.sources.PythonFileSource >>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = sage.doctest.sources.__file__>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> type(FDS)<class sage.doctest.sources.PythonFileSource >ending_docstring(line)
Determines whether the input line ends a docstring.
INPUT:• line – a string, one line of an input file.
OUTPUT:• an object that, when evaluated in a boolean context, gives True or False depending on whether the
input line marks the end of a docstring.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.util import NestedName
sage: filename = sage.doctest.sources.__file__
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS._init()
sage: FDS.quotetype = ""sage: FDS.ending_docstring("")<...Match object...>sage: FDS.ending_docstring( \"\"\" )>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.util import NestedName>>> filename = sage.doctest.sources.__file__>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS._init()>>> FDS.quotetype = "">>> FDS.ending_docstring("")<...Match object...>>>> FDS.ending_docstring( \"\"\" )start_finish_can_overlap = False
starting_docstring(line)
Determines whether the input line starts a docstring.
If the input line does start a docstring (a triple quote), then this function updates self.qualified_name.
26

Chapter 2. Classes for sources of doctests

Sage's Doctesting Framework, Release 10.4

INPUT:• line – a string, one line of an input file
OUTPUT:• either None or a Match object.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.util import NestedName
sage: filename = sage.doctest.sources.__file__
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS._init()
sage: FDS.starting_docstring("r")<...Match object...>sage: FDS.ending_docstring("")<...Match object...>sage: FDS.qualified_name = NestedName(FDS.basename)sage: FDS.starting_docstring("class MyClass():")
sage: FDS.starting_docstring("
def hello_world(self):")sage: FDS.starting_docstring("")<...Match object...>sage: FDS.qualified_namesage.doctest.sources.MyClass.hello_world
sage: FDS.ending_docstring("")<...Match object...>sage: FDS.starting_docstring("class NewClass():")
sage: FDS.starting_docstring("")<...Match object...>sage: FDS.ending_docstring("")<...Match object...>sage: FDS.qualified_namesage.doctest.sources.NewClass
sage: FDS.starting_docstring("print(")
sage: FDS.starting_docstring("
Not a docstring")sage: FDS.starting_docstring(")")
sage: FDS.starting_docstring("def foo():")
sage: FDS.starting_docstring("
This is a docstring")<...Match object...>>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.util import NestedName>>> filename = sage.doctest.sources.__file__>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS._init()>>> FDS.starting_docstring("r")<...Match object...>>>> FDS.ending_docstring("")<...Match object...>>>> FDS.qualified_name = NestedName(FDS.basename)>>> FDS.starting_docstring("class MyClass():")>>> FDS.starting_docstring("
def hello_world(self):")>>> FDS.starting_docstring("")<...Match object...>(continues on next page)

27

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> FDS.qualified_namesage.doctest.sources.MyClass.hello_world>>> FDS.ending_docstring("")<...Match object...>>>> FDS.starting_docstring("class NewClass():")>>> FDS.starting_docstring("")<...Match object...>>>> FDS.ending_docstring("")<...Match object...>>>> FDS.qualified_namesage.doctest.sources.NewClass>>> FDS.starting_docstring("print(")>>> FDS.starting_docstring("
Not a docstring")>>> FDS.starting_docstring(")")>>> FDS.starting_docstring("def foo():")>>> FDS.starting_docstring("
This is a docstring<...Match object...>")class sage.doctest.sources.RestSource
Bases: SourceLanguage
This class defines the functions needed for the extraction of doctests from ReST sources.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: type(FDS)<class sage.doctest.sources.RestFileSource >>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = "sage_doc.rst">>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> type(FDS)<class sage.doctest.sources.RestFileSource >ending_docstring(line)
When the indentation level drops below the initial level the block ends.
INPUT:• line – a string, one line of an input file
OUTPUT:• a boolean, whether the verbatim block is ending.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS._init()(continues on next page)

28

Chapter 2. Classes for sources of doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: FDS.starting_docstring("Hello world::")True
sage: FDS.ending_docstring("sage: 2 + 2")
False
sage: FDS.ending_docstring("
4")
False
sage: FDS.ending_docstring("We are now done")True>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = "sage_doc.rst">>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS._init()>>> FDS.starting_docstring("Hello world::")True>>> FDS.ending_docstring("sage: 2 + 2")
False>>> FDS.ending_docstring("
4")
False
>>> FDS.ending_docstring("We are now done")True

parse_docstring(docstring, namespace, start)
Return a list of doctest defined in this docstring.
Code blocks in a REST file can contain python functions with their own docstrings in addition to in-line
doctests. We want to include the tests from these inner docstrings, but Python’s doctesting module has aproblem if we just pass on the whole block, since it expects to get just a docstring, not the Python code as
well.
Our solution is to create a new doctest source from this code block and append the doctests created from thatsource. We then replace the occurrences of “sage:” and “>>>” occurring inside a triple quote with “safe:” sothat the doctest module doesn’t treat them as tests.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.parsing import SageDocTestParser
sage: from sage.doctest.util import NestedName
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS.parser = SageDocTestParser(set([ sage ]))sage: FDS.qualified_name = NestedName( sage_doc )sage: s = "Some text::\n\ndef example_python_function(a, \
....:
b):\n
\n
Brief description \....:of function.\n\nEXAMPLES::\n\n\....:sage: test1()\nsage: test2()\n
\
....:
\n
return a + b\n\n
sage: test3()\n\nMore \....:ReST documentation."sage: tests = FDS.parse_docstring(s, {}, 100)
sage: len(tests)2
sage: for ex in tests[0].examples:....:
print(ex.sage_source)(continues on next page)

29

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

test3()
sage: for ex in tests[1].examples:....:
print(ex.sage_source)
test1()
test2()
sig_on_count() # check sig_on/off pairings (virtual doctest)>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.parsing import SageDocTestParser>>> from sage.doctest.util import NestedName>>> filename = "sage_doc.rst">>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS.parser = SageDocTestParser(set([ sage ]))>>> FDS.qualified_name = NestedName( sage_doc )>>> s = "Some text::\n\ndef example_python_function(a, \
...
b):\n
\n
Brief description \...
of function.\n\nEXAMPLES::\n\n\...
sage: test1()\nsage: test2()\n\...
\nreturn a + b\n\n
sage: test3()\n\nMore \...
ReST documentation.">>> tests = FDS.parse_docstring(s, {}, Integer(100))>>> len(tests)
2>>> for ex in tests[Integer(0)].examples:...
print(ex.sage_source)
test3()>>> for ex in tests[Integer(1)].examples:...
print(ex.sage_source)
test1()
test2()
sig_on_count() # check sig_on/off pairings (virtual doctest)

start_finish_can_overlap = True
starting_docstring(line)
A line ending with a double colon starts a verbatim block in a ReST file, as does a line containing ..
CODE-BLOCK:: language.
This function also determines whether the docstring block should be joined with the previous one, or should
be skipped.
INPUT:• line – a string, one line of an input file
OUTPUT:• either None or a Match object.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS._init()(continues on next page)

30

Chapter 2. Classes for sources of doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: FDS.starting_docstring("Hello world::")True
sage: FDS.ending_docstring("sage: 2 + 2")
False
sage: FDS.ending_docstring("
4")
False
sage: FDS.ending_docstring("We are now done")True
sage: FDS.starting_docstring(".. link")
sage: FDS.starting_docstring("::")
True
sage: FDS.linking
True>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = "sage_doc.rst">>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS._init()>>> FDS.starting_docstring("Hello world::")True>>> FDS.ending_docstring("sage: 2 + 2")
False>>> FDS.ending_docstring("
4")
False
>>> FDS.ending_docstring("We are now done")
True>>> FDS.starting_docstring(".. link")>>> FDS.starting_docstring("::")
True>>> FDS.linking
Trueclass sage.doctest.sources.SourceLanguage
Bases: object
An abstract class for functions that depend on the programming language of a doctest source.
Currently supported languages include Python, ReST and LaTeX.
parse_docstring(docstring, namespace, start)
Return a list of doctest defined in this docstring.
This function is called by DocTestSource._process_doc(). The default implementation, defined
here, is to use the sage.doctest.parsing.SageDocTestParser attached to this source to get
doctests from the docstring.
INPUT:• docstring – a string containing documentation and tests.
• namespace – a dictionary or sage.doctest.util.RecordingDict.
• start – an integer, one less than the starting line number
EXAMPLES:

31

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.parsing import SageDocTestParser
sage: from sage.doctest.util import NestedName
sage: filename = sage.doctest.util.__file__
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: doctests, _ = FDS.create_doctests({})
sage: for dt in doctests:....:
FDS.qualified_name = dt.name....:dt.examples = dt.examples[:-1] # strip off the sig_on() test....:
assert(FDS.parse_docstring(dt.docstring,{},dt.lineno-1)[0] == dt)>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.parsing import SageDocTestParser>>> from sage.doctest.util import NestedName>>> filename = sage.doctest.util.__file__>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> doctests, _ = FDS.create_doctests({})>>> for dt in doctests:...
FDS.qualified_name = dt.name...
dt.examples = dt.examples[:-Integer(1)] # strip off the sig_on() test...
assert(FDS.parse_docstring(dt.docstring,{},dt.lineno˓→Integer(1))[Integer(0)] == dt)class sage.doctest.sources.StringDocTestSource(basename, source, options, printpath,lineno_shift=0)
Bases: DocTestSourceThis class creates doctests from a string.
INPUT:• basename – string such as ‘sage.doctests.sources’, going into the names of created doctests and examples.
• source – a string, giving the source code to be parsed for doctests.
• options – a sage.doctest.control.DocTestDefaults or equivalent.
• printpath – a string, to be used in place of a filename when doctest failures are displayed.
• lineno_shift – an integer (default: 0) by which to shift the line numbers of all doctests defined in thisstring.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import StringDocTestSource, PythonSource
sage: from sage.structure.dynamic_class import dynamic_classsage: s = "\nsage: 2 + 2\n4\n"sage: PythonStringSource = dynamic_class( PythonStringSource ,˓→(StringDocTestSource, PythonSource))
sage: PSS = PythonStringSource( <runtime> , s, DocTestDefaults(), runtime )sage: dt, extras = PSS.create_doctests({})
sage: len(dt)1sage: extras[ tab ][](continues on next page)

32

Chapter 2. Classes for sources of doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: extras[ line_number ]False
sage: s = "\n\tsage: 2 + 2\n\t4\n"sage: PSS = PythonStringSource( <runtime> , s, DocTestDefaults(),
sage: dt, extras = PSS.create_doctests({})
sage: extras[ tab ][ 2 , 3 ]sage: s = "\nsage: import warnings; warnings.warn( foo )\n˓→UserWarning: foo \n"sage: PSS = PythonStringSource( <runtime> , s, DocTestDefaults(),
sage: dt, extras = PSS.create_doctests({})
sage: extras[ line_number ]Trueruntime )doctest:1:␣
runtime )>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import StringDocTestSource, PythonSource>>> from sage.structure.dynamic_class import dynamic_class>>> s = "\nsage: 2 + 2\n4\n">>> PythonStringSource = dynamic_class( PythonStringSource ,(StringDocTestSource,␣
˓→PythonSource))>>> PSS = PythonStringSource( <runtime> , s, DocTestDefaults(), runtime )>>> dt, extras = PSS.create_doctests({})>>> len(dt)1>>> extras[ tab ][]>>> extras[ line_number ]False>>> s = "\n\tsage: 2 + 2\n\t4\n">>> PSS = PythonStringSource( <runtime> , s, DocTestDefaults(),>>> dt, extras = PSS.create_doctests({})>>> extras[ tab ][ 2 , 3 ]>>> s = "\nsage: import warnings; warnings.warn( foo )\n˓→UserWarning: foo \n">>> PSS = PythonStringSource( <runtime> , s, DocTestDefaults(),>>> dt, extras = PSS.create_doctests({})>>> extras[ line_number ]Trueruntime )doctest:1:␣
runtime )create_doctests(namespace)
Creates doctests from this string.
INPUT:• namespace – a dictionary or sage.doctest.util.RecordingDict.
OUTPUT:• doctests – a list of doctests defined by this string• tab_locations – either False or a list of linenumbers on which tabs appear.
EXAMPLES:
33

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import StringDocTestSource, PythonSource
sage: from sage.structure.dynamic_class import dynamic_classsage: s = "\nsage: 2 + 2\n4\n"sage: PythonStringSource = dynamic_class( PythonStringSource ,˓→(StringDocTestSource, PythonSource))
sage: PSS = PythonStringSource( <runtime> , s, DocTestDefaults(), runtime )sage: dt, tabs = PSS.create_doctests({})
sage: for t in dt:....:
print("{} {}".format(t.name, t.examples[0].sage_source))<runtime> 2 + 2>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import StringDocTestSource, PythonSource>>> from sage.structure.dynamic_class import dynamic_class>>> s = "\nsage: 2 + 2\n4\n">>> PythonStringSource = dynamic_class( PythonStringSource ,˓→(StringDocTestSource, PythonSource))>>> PSS = PythonStringSource( <runtime> , s, DocTestDefaults(), runtime )>>> dt, tabs = PSS.create_doctests({})>>> for t in dt:...
print("{} {}".format(t.name, t.examples[Integer(0)].sage_source))<runtime> 2 + 2class sage.doctest.sources.TexSource
Bases: SourceLanguage
This class defines the functions needed for the extraction of doctests from a LaTeX source.
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_paper.tex"
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: type(FDS)<class sage.doctest.sources.TexFileSource >>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = "sage_paper.tex">>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> type(FDS)<class sage.doctest.sources.TexFileSource >ending_docstring(line, check_skip=True)
Determines whether the input line ends a docstring.
Docstring blocks in tex files are defined by verbatim or lstlisting environments, and can be linked together byadding %link immediately after the end{verbatim} or end{lstlisting}.
Within a verbatim (or lstlisting) block, you can tell Sage not to process the rest of the block by including a%skip line.
INPUT:• line – a string, one line of an input file
34

Chapter 2. Classes for sources of doctests

Sage's Doctesting Framework, Release 10.4

• check_skip – boolean (default: True), used internally in starting_docstring.
OUTPUT:• a boolean giving whether the input line marks the end of a docstring (verbatim block).
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_paper.tex"
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS._init()
sage: FDS.ending_docstring(r"\end{verbatim}")True
sage: FDS.ending_docstring(r"\end{lstlisting}")True
sage: FDS.linking
False>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = "sage_paper.tex">>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS._init()>>> FDS.ending_docstring(r"\end{verbatim}")True>>> FDS.ending_docstring(r"\end{lstlisting}")
True>>> FDS.linking
False

Use %link to link with the next verbatim block:
sage: FDS.ending_docstring(r"\end{verbatim}%link")
True
sage: FDS.linking
True>>> from sage.all import *>>> FDS.ending_docstring(r"\end{verbatim}%link")
True>>> FDS.linking
True%skip also ends a docstring block:
sage: FDS.ending_docstring("%skip")True>>> from sage.all import *>>> FDS.ending_docstring("%skip")
Truestart_finish_can_overlap = False

35

Sage's Doctesting Framework, Release 10.4

starting_docstring(line)
Determines whether the input line starts a docstring.
Docstring blocks in tex files are defined by verbatim or lstlisting environments, and can be linked together byadding %link immediately after the end{verbatim} or end{lstlisting}.
Within a verbatim (or lstlisting) block, you can tell Sage not to process the rest of the block by including a%skip line.
INPUT:• line – a string, one line of an input file
OUTPUT:• a boolean giving whether the input line marks the start of a docstring (verbatim block).
EXAMPLES:
sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_paper.tex"
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS._init()>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> filename = "sage_paper.tex">>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> FDS._init()

We start docstrings with begin{verbatim} or begin{lstlisting}:sage: FDS.starting_docstring(r"\begin{verbatim}")True
sage: FDS.starting_docstring(r"\begin{lstlisting}")True
sage: FDS.skipping
False
sage: FDS.ending_docstring("sage: 2+2")
False
sage: FDS.ending_docstring("4")
False>>> from sage.all import *>>> FDS.starting_docstring(r"\begin{verbatim}")True>>> FDS.starting_docstring(r"\begin{lstlisting}")
True>>> FDS.skipping
False>>> FDS.ending_docstring("sage: 2+2")
False>>> FDS.ending_docstring("4")
False

To start ignoring the rest of the verbatim block, use %skip:

36

Chapter 2. Classes for sources of doctests

Sage's Doctesting Framework, Release 10.4

sage: FDS.ending_docstring("%skip")Truesage: FDS.skippingTruesage: FDS.starting_docstring("sage: raise RuntimeError")False>>> from sage.all import *>>> FDS.ending_docstring("%skip")
True>>> FDS.skipping
True>>> FDS.starting_docstring("sage: raise RuntimeError")False

You can even pretend to start another verbatim block while skipping:sage: FDS.starting_docstring(r"\begin{verbatim}")False
sage: FDS.skippingTrue>>> from sage.all import *>>> FDS.starting_docstring(r"\begin{verbatim}")False>>> FDS.skipping
True

To stop skipping end the verbatim block:
sage: FDS.starting_docstring(r"\end{verbatim} %link")
False
sage: FDS.skipping
False>>> from sage.all import *>>> FDS.starting_docstring(r"\end{verbatim} %link")
False>>> FDS.skipping
False

Linking works even when the block was ended while skipping:sage: FDS.linking
True
sage: FDS.starting_docstring(r"\begin{verbatim}")True>>> from sage.all import *>>> FDS.linking
True>>> FDS.starting_docstring(r"\begin{verbatim}")True

sage.doctest.sources.get_basename(path)
This function returns the basename of the given path, e.g. sage.doctest.sources or doc.ru.tutorial.tour_advanced37

Sage's Doctesting Framework, Release 10.4

EXAMPLES:
sage: from sage.doctest.sources import get_basename
sage: import os
sage: get_basename(sage.doctest.sources.__file__)sage.doctest.sources
sage: get_basename(os.path.join(sage.structure.__path__[0],sage.structure.element.pxd

element.pxd ))>>> from sage.all import *>>> from sage.doctest.sources import get_basename>>> import os>>> get_basename(sage.doctest.sources.__file__)sage.doctest.sources>>> get_basename(os.path.join(sage.structure.__path__[Integer(0)],sage.structure.element.pxd

38element.pxd ))Chapter 2. Classes for sources of doctests

CHAPTER

THREE

PROCESSES FOR RUNNING DOCTESTSThis module controls the processes started by Sage that actually run the doctests.
EXAMPLES:
The following examples are used in doctesting this file:
sage: doctest_var = 42; doctest_var^2
1764
sage: R.<a> = ZZ[]
sage: a + doctest_var
a + 42>>> from sage.all import *>>> doctest_var = Integer(42); doctest_var**Integer(2)1764>>> R = ZZ[ a ]; (a,) = R._first_ngens(1)>>> a + doctest_var
a + 42AUTHORS:• David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
• Jeroen Demeyer (2013 and 2015) – major improvements to forking and logging
class sage.doctest.forker.DocTestDispatcher(controller)
Bases: SageObject
Creates parallel DocTestWorker processes and dispatches doctesting tasks.
dispatch()
Run the doctests for the controller’s specified sources, by calling parallel_dispatch() or serial_dispatch() according to the --serial option.
EXAMPLES:
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.doctest.forker import DocTestDispatcher
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.util import Timersage: import os
sage: freehom = os.path.join(SAGE_SRC, sage , modules , free_module_
˓→homspace.py )sage: bigo = os.path.join(SAGE_SRC, sage , rings , big_oh.py )sage: DC = DocTestController(DocTestDefaults(), [freehom, bigo])
sage: DC.expand_files_into_sources()(continues on next page)

39

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: DD = DocTestDispatcher(DC)sage: DR = DocTestReporter(DC)sage: DC.reporter = DRsage: DC.dispatcher = DD
sage: DC.timer = Timer().start()sage: DD.dispatch()sage -t .../sage/modules/free_module_homspace.py[... tests, ... s]sage -t .../sage/rings/big_oh.py[... tests, ... s]>>> from sage.all import *>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> from sage.doctest.forker import DocTestDispatcher>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.util import Timer>>> import os>>> freehom = os.path.join(SAGE_SRC, sage , modules , free_module_homspace.
˓→py )>>> bigo = os.path.join(SAGE_SRC, sage , rings , big_oh.py )>>> DC = DocTestController(DocTestDefaults(), [freehom, bigo])>>> DC.expand_files_into_sources()>>> DD = DocTestDispatcher(DC)>>> DR = DocTestReporter(DC)>>> DC.reporter = DR>>> DC.dispatcher = DD>>> DC.timer = Timer().start()>>> DD.dispatch()sage -t .../sage/modules/free_module_homspace.py[... tests, ... s]sage -t .../sage/rings/big_oh.py[... tests, ... s]parallel_dispatch()
Run the doctests from the controller’s specified sources in parallel.
This creates DocTestWorker subprocesses, while the master process checks for timeouts and collects anddisplays the results.
EXAMPLES:
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.doctest.forker import DocTestDispatcher
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.util import Timersage: import os
sage: crem = os.path.join(SAGE_SRC, sage , databases , cremona.py )sage: bigo = os.path.join(SAGE_SRC, sage , rings , big_oh.py )sage: DC = DocTestController(DocTestDefaults(), [crem, bigo])
sage: DC.expand_files_into_sources()sage: DD = DocTestDispatcher(DC)sage: DR = DocTestReporter(DC)sage: DC.reporter = DRsage: DC.dispatcher = DD
sage: DC.timer = Timer().start()sage: DD.parallel_dispatch()sage -t .../databases/cremona.py(continues on next page)

40

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)[... tests, ... s]sage -t .../rings/big_oh.py[... tests, ... s]>>> from sage.all import *>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> from sage.doctest.forker import DocTestDispatcher>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.util import Timer>>> import os>>> crem = os.path.join(SAGE_SRC, sage , databases , cremona.py )>>> bigo = os.path.join(SAGE_SRC, sage , rings , big_oh.py )>>> DC = DocTestController(DocTestDefaults(), [crem, bigo])>>> DC.expand_files_into_sources()>>> DD = DocTestDispatcher(DC)>>> DR = DocTestReporter(DC)>>> DC.reporter = DR>>> DC.dispatcher = DD>>> DC.timer = Timer().start()>>> DD.parallel_dispatch()sage -t .../databases/cremona.py[... tests, ... s]sage -t .../rings/big_oh.py[... tests, ... s]

If the exitfirst=True option is given, the results for a failing module will be immediately printed andany other ongoing tests canceled:sage: from tempfile import NamedTemporaryFile as NTF
sage: with NTF(suffix=".py", mode="w+t") as f1, \....:NTF(suffix=".py", mode="w+t") as f2:....:_ = f1.write("\nsage: import time; time.sleep(60)\n")....:
f1.flush()....:_ = f2.write("\nsage: True\nFalse\n")....:f2.flush()....:
DC = DocTestController(DocTestDefaults(exitfirst=True,....:
nthreads=2),....:[f1.name, f2.name])....:
DC.expand_files_into_sources()....:
DD = DocTestDispatcher(DC)....:DR = DocTestReporter(DC)....:DC.reporter = DR....:DC.dispatcher = DD....:
DC.timer = Timer().start()....:
DD.parallel_dispatch()sage -t ...
**********************************************************************
File "...", line 2, in ...
Failed example:
True
Expected:
False
Got:
True
**********************************************************************
1 item had failures:(continues on next page)

41

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

1 of
1 in ...
[1 test, 1 failure, ... s]
Killing test ...
>>> from sage.all import *>>> from tempfile import NamedTemporaryFile as NTF>>> with NTF(suffix=".py", mode="w+t") as f1,
NTF(suffix=".py", mode="w+t˓→") as f2:...
_ = f1.write("\nsage: import time; time.sleep(60)\n")...
f1.flush()...
_ = f2.write("\nsage: True\nFalse\n")...
f2.flush()...
DC = DocTestController(DocTestDefaults(exitfirst=True,...
nthreads=Integer(2)),...
[f1.name, f2.name])...
DC.expand_files_into_sources()...
DD = DocTestDispatcher(DC)...
DR = DocTestReporter(DC)...
DC.reporter = DR...
DC.dispatcher = DD...
DC.timer = Timer().start()...
DD.parallel_dispatch()sage -t ...
**********************************************************************
File "...", line 2, in ...
Failed example:
True
Expected:
False
Got:
True
**********************************************************************
1 item had failures:
1 of
1 in ...
[1 test, 1 failure, ... s]
Killing test ...

serial_dispatch()
Run the doctests from the controller’s specified sources in series.
There is no graceful handling for signals, no possibility of interrupting tests and no timeout.
EXAMPLES:
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.doctest.forker import DocTestDispatcher
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.util import Timersage: import os
sage: homset = os.path.join(SAGE_SRC, sage , rings , homset.py )sage: ideal = os.path.join(SAGE_SRC, sage , rings , ideal.py )sage: DC = DocTestController(DocTestDefaults(), [homset, ideal])
sage: DC.expand_files_into_sources()sage: DD = DocTestDispatcher(DC)sage: DR = DocTestReporter(DC)sage: DC.reporter = DRsage: DC.dispatcher = DD
(continues on next page)

42

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: DC.timer = Timer().start()sage: DD.serial_dispatch()sage -t .../rings/homset.py[... tests, ... s]sage -t .../rings/ideal.py[... tests, ... s]>>> from sage.all import *>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> from sage.doctest.forker import DocTestDispatcher>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.util import Timer>>> import os>>> homset = os.path.join(SAGE_SRC, sage , rings , homset.py )>>> ideal = os.path.join(SAGE_SRC, sage , rings , ideal.py )>>> DC = DocTestController(DocTestDefaults(), [homset, ideal])>>> DC.expand_files_into_sources()>>> DD = DocTestDispatcher(DC)>>> DR = DocTestReporter(DC)>>> DC.reporter = DR>>> DC.dispatcher = DD>>> DC.timer = Timer().start()>>> DD.serial_dispatch()sage -t .../rings/homset.py[... tests, ... s]sage -t .../rings/ideal.py[... tests, ... s]class sage.doctest.forker.DocTestTask(source)
Bases: object
This class encapsulates the tests from a single source.
This class does not insulate from problems in the source (e.g. entering an infinite loop or causing a segfault), thathas to be dealt with at a higher level.
INPUT:• source – a sage.doctest.sources.DocTestSource instance.
• verbose – boolean, controls reporting of progress by doctest.DocTestRunner.
EXAMPLES:
sage: from sage.doctest.forker import DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: import os
sage: filename = sage.doctest.sources.__file__
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename, DD)
sage: DTT = DocTestTask(FDS)
sage: DC = DocTestController(DD,[filename])
sage: ntests, results = DTT(options=DD)
sage: ntests >= 300 or ntests
Truesage: sorted(results.keys())[ cputime , err , failures , optionals , tests , walltime , walltime_skips ]43

Sage's Doctesting Framework, Release 10.4>>> from sage.all import *>>> from sage.doctest.forker import DocTestTask>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.control import DocTestDefaults, DocTestController>>> import os>>> filename = sage.doctest.sources.__file__>>> DD = DocTestDefaults()>>> FDS = FileDocTestSource(filename, DD)>>> DTT = DocTestTask(FDS)>>> DC = DocTestController(DD,[filename])>>> ntests, results = DTT(options=DD)>>> ntests >= Integer(300) or ntests
True>>> sorted(results.keys())[ cputime , err , failures , optionals , tests , walltime , walltime_skips ]class sage.doctest.forker.DocTestWorker(source, options, funclist=[], baseline=None)
Bases: Process
The DocTestWorker process runs one DocTestTask for a given source. It returns messages about doctest failures(or all tests if verbose doctesting) through a pipe and returns results through a multiprocessing.Queue
instance (both these are created in the start() method).
It runs the task in its own process-group, such that killing the process group kills this process together with its childprocesses.
The class has additional methods and attributes for bookkeeping by the master process. Except in run(), nothingfrom this class should be accessed by the child process.
INPUT:• source – a DocTestSource instance• options – an object representing doctest options.
• funclist – a list of callables to be called at the start of the child process.
• baseline – dictionary, the baseline_stats value
EXAMPLES:
sage: from sage.doctest.forker import DocTestWorker, DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: filename = sage.doctest.util.__file__
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename, DD)
sage: W = DocTestWorker(FDS, DD)
sage: W.start()sage: DC = DocTestController(DD, filename)sage: reporter = DocTestReporter(DC)sage: W.join() # Wait for worker to finish
sage: result = W.result_queue.get()sage: reporter.report(FDS, False, W.exitcode, result, "")[... tests, ... s]>>> from sage.all import *>>> from sage.doctest.forker import DocTestWorker, DocTestTask(continues on next page)

44

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> filename = sage.doctest.util.__file__>>> DD = DocTestDefaults()>>> FDS = FileDocTestSource(filename, DD)>>> W = DocTestWorker(FDS, DD)>>> W.start()>>> DC = DocTestController(DD, filename)>>> reporter = DocTestReporter(DC)>>> W.join() # Wait for worker to finish>>> result = W.result_queue.get()>>> reporter.report(FDS, False, W.exitcode, result, "")[... tests, ... s]kill()
Kill this worker. Return True if the signal(s) are sent successfully or False if the worker process no longer
exists.
This method is only called if there is something wrong with the worker. Under normal circumstances, theworker is supposed to exit by itself after finishing.
The first time this is called, use SIGQUIT. This will trigger the cysignals SIGQUIT handler and try to print
an enhanced traceback.
Subsequent times, use SIGKILL. Also close the message pipe if it was still open.
EXAMPLES:
sage: import time
sage: from sage.doctest.forker import DocTestWorker, DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: filename = os.path.join(SAGE_SRC, sage , doctest , tests , 99seconds.rst
˓→ )sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename, DD)>>> from sage.all import *>>> import time>>> from sage.doctest.forker import DocTestWorker, DocTestTask>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> filename = os.path.join(SAGE_SRC, sage , doctest , tests , 99seconds.rst )>>> DD = DocTestDefaults()>>> FDS = FileDocTestSource(filename, DD)

We set up the worker to start by blocking SIGQUIT, such that killing will fail initially:
sage: from cysignals.pselect import PSelecter
sage: import signal
sage: def block_hup():....:# We never __exit__()....:PSelecter([signal.SIGQUIT]).__enter__()sage: W = DocTestWorker(FDS, DD, [block_hup])(continues on next page)

45

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: W.start()sage: W.killed
False
sage: W.kill()True
sage: W.killed
True
sage: time.sleep(float(0.2)) # Worker doesn t die
sage: W.kill()# Worker dies now
True
sage: time.sleep(1)sage: W.is_alive()
False>>> from sage.all import *>>> from cysignals.pselect import PSelecter>>> import signal>>> def block_hup():...
# We never __exit__()...
PSelecter([signal.SIGQUIT]).__enter__()>>> W = DocTestWorker(FDS, DD, [block_hup])>>> W.start()>>> W.killed
False>>> W.kill()
True>>> W.killed
True>>> time.sleep(float(RealNumber( 0.2 ))) # Worker doesn t die>>> W.kill()# Worker dies now
True>>> time.sleep(Integer(1))>>> W.is_alive()
False

read_messages()
In the master process, read from the pipe and store the data read in the messages attribute.
Note: This function may need to be called multiple times in order to read all of the messages.
EXAMPLES:
sage: from sage.doctest.forker import DocTestWorker, DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: filename = sage.doctest.util.__file__
sage: DD = DocTestDefaults(verbose=True,nthreads=2)
sage: FDS = FileDocTestSource(filename, DD)
sage: W = DocTestWorker(FDS, DD)
sage: W.start()sage: while W.rmessages is not None:....:
W.read_messages()sage: W.join()(continues on next page)46

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: len(W.messages) > 0
True>>> from sage.all import *>>> from sage.doctest.forker import DocTestWorker, DocTestTask>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> filename = sage.doctest.util.__file__>>> DD = DocTestDefaults(verbose=True,nthreads=Integer(2))>>> FDS = FileDocTestSource(filename, DD)>>> W = DocTestWorker(FDS, DD)>>> W.start()>>> while W.rmessages is not None:...
W.read_messages()>>> W.join()>>> len(W.messages) > Integer(0)
True

run()
Runs the DocTestTask under its own PGID.
save_result_output()
Annotate self with self.result (the result read through the result_queue and with self.
output, the complete contents of self.outtmpfile.
Then close the Queue and self.
outtmpfile.
EXAMPLES:
sage: from sage.doctest.forker import DocTestWorker, DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: filename = sage.doctest.util.__file__
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename, DD)
sage: W = DocTestWorker(FDS, DD)
sage: W.start()sage: W.join()sage: W.save_result_output()sage: sorted(W.result[1].keys())[ cputime , err , failures , optionals , tests , walltime , walltime_
˓→skips ]sage: len(W.output) > 0
True>>> from sage.all import *>>> from sage.doctest.forker import DocTestWorker, DocTestTask>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> filename = sage.doctest.util.__file__>>> DD = DocTestDefaults()>>> FDS = FileDocTestSource(filename, DD)>>> W = DocTestWorker(FDS, DD)>>> W.start()(continues on next page)

47

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> W.join()>>> W.save_result_output()>>> sorted(W.result[Integer(1)].keys())[ cputime , err , failures , optionals ,˓→skips ]>>> len(W.output) > Integer(0)Truetests ,

walltime ,walltime_

Note: This method is called from the parent process, not from the subprocess.
start()Start the worker and close the writing end of the message pipe.
class sage.doctest.forker.SageDocTestRunner(*args, **kwds)
Bases: DocTestRunner
A customized version of DocTestRunner that tracks dependencies of doctests.
INPUT:• stdout – an open file to restore for debugging• checker – None, or an instance of doctest.OutputChecker• verbose – boolean, determines whether verbose printing is enabled.
• optionflags – Controls the comparison with the expected output. See testmod for more information.
• baseline – dictionary, the baseline_stats value
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD,
˓→ optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: DTR
<sage.doctest.forker.SageDocTestRunner object at ...>>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> import doctest, sys, os>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD,␣
˓→optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> DTR<sage.doctest.forker.SageDocTestRunner object at ...>

compile_and_execute(example, compiler, globs)
Runs the given example, recording dependencies.
Rather than using a basic dictionary, Sage’s doctest runner uses a sage.doctest.util.
RecordingDict, which records every time a value is set or retrieved. Executing the given code withthis recording dictionary as the namespace allows Sage to track dependencies between doctest lines. For
example, in the following two lines
48

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

sage: R.<x> = ZZ[]
sage: f = x^2 + 1>>> from sage.all import *>>> R = ZZ[ x ]; (x,) = R._first_ngens(1)>>> f = x**Integer(2) + Integer(1)the recording dictionary records that the second line depends on the first since the first INSERTS x into theglobal namespace and the second line RETRIEVES x from the global namespace.
INPUT:• example – a doctest.Example instance.
• compiler – a callable that, applied to example, produces a code object• globs – a dictionary in which to execute the code.
OUTPUT:• the output of the compiled code snippet.
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.util import RecordingDictsage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os, hashlib
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD,....:
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: DTR.running_doctest_digest = hashlib.md5()
sage: filename = sage.doctest.forker.__file__
sage: FDS = FileDocTestSource(filename, DD)
sage: globs = RecordingDict(globals())
sage: doctest_var in globs
False
sage: doctests, extras = FDS.create_doctests(globs)
sage: ex0 = doctests[0].examples[0]sage: flags = 32768 if sys.version_info.minor < 8 else 524288
sage: def compiler(ex):....:return compile(ex.source, <doctest sage.doctest.forker[0]> ,....:single , flags, 1)
sage: DTR.compile_and_execute(ex0, compiler, globs)
1764
sage: globs[ doctest_var ]42
sage: globs.set{ doctest_var }sage: globs.got{ Integer }>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.util import RecordingDict(continues on next page)

49

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> import doctest, sys, os, hashlib>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD,...
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> DTR.running_doctest_digest = hashlib.md5()>>> filename = sage.doctest.forker.__file__>>> FDS = FileDocTestSource(filename, DD)>>> globs = RecordingDict(globals())>>> doctest_var in globs
False>>> doctests, extras = FDS.create_doctests(globs)>>> ex0 = doctests[Integer(0)].examples[Integer(0)]>>> flags = Integer(32768) if sys.version_info.minor < Integer(8) else␣
˓→Integer(524288)>>> def compiler(ex):...
return compile(ex.source, <doctest sage.doctest.forker[0]> ,...
single , flags, Integer(1))>>> DTR.compile_and_execute(ex0, compiler, globs)
1764>>> globs[ doctest_var ]42>>> globs.set{ doctest_var }>>> globs.got{ Integer }Now we can execute some more doctests to see the dependencies.
sage: ex1 = doctests[0].examples[1]sage: def compiler(ex):....:return compile(ex.source, <doctest sage.doctest.forker[1]> ,....:single , flags, 1)
sage: DTR.compile_and_execute(ex1, compiler, globs)
sage: sorted(list(globs.set))[ R , a ]sage: globs.got{ ZZ }
sage: ex1.predecessors[]>>> from sage.all import *>>> ex1 = doctests[Integer(0)].examples[Integer(1)]>>> def compiler(ex):...
return compile(ex.source, <doctest sage.doctest.forker[1]> ,...
single , flags, Integer(1))>>> DTR.compile_and_execute(ex1, compiler, globs)>>> sorted(list(globs.set))[ R , a ]>>> globs.got{ ZZ }>>> ex1.predecessors[]
sage: ex2 = doctests[0].examples[2]sage: def compiler(ex):(continues on next page)

50

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)....:
return compile(ex.source, <doctest sage.doctest.forker[2]> ,....:single , flags, 1)
sage: DTR.compile_and_execute(ex2, compiler, globs)
a + 42sage: list(globs.set)[]
sage: sorted(list(globs.got))[ a , doctest_var ]sage: set(ex2.predecessors) == set([ex0,ex1])
True>>> from sage.all import *>>> ex2 = doctests[Integer(0)].examples[Integer(2)]>>> def compiler(ex):...
return compile(ex.source, <doctest sage.doctest.forker[2]> ,...
single , flags, Integer(1))>>> DTR.compile_and_execute(ex2, compiler, globs)
a + 42>>> list(globs.set)[]>>> sorted(list(globs.got))[ a , doctest_var ]>>> set(ex2.predecessors) == set([ex0,ex1])
Truereport_failure(out, test, example, got, globs)
Called when a doctest fails.
INPUT:• out – a function for printing• test – a doctest.DocTest instance• example – a doctest.Example instance in test
• got – a string, the result of running example• globs – a dictionary of globals, used if in debugging mode
OUTPUT:• prints a report to out
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = sage.doctest.forker.__file__
sage: FDS = FileDocTestSource(filename, DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: ex = doctests[0].examples[0]sage: DTR.no_failure_yet = True
sage: DTR.report_failure(sys.stdout.write, doctests[0], ex, BAD ANSWER\n , {}(continues on next page)

51

Sage's Doctesting Framework, Release 10.4

(continued from previous page))**********************************************************************
File ".../sage/doctest/forker.py", line 12, in sage.doctest.forker
Failed example:
doctest_var = 42; doctest_var^2
Expected:
1764
Got:
BAD ANSWER
˓→>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> import doctest, sys, os>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> filename = sage.doctest.forker.__file__>>> FDS = FileDocTestSource(filename, DD)>>> doctests, extras = FDS.create_doctests(globals())>>> ex = doctests[Integer(0)].examples[Integer(0)]>>> DTR.no_failure_yet = True>>> DTR.report_failure(sys.stdout.write, doctests[Integer(0)], ex, BAD␣˓→ANSWER\n , {})**********************************************************************
File ".../sage/doctest/forker.py", line 12, in sage.doctest.forker
Failed example:
doctest_var = 42; doctest_var^2
Expected:
1764
Got:
BAD ANSWER

If debugging is turned on this function starts an IPython prompt when a test returns an incorrect answer:
sage: sage0.quit()
sage: _ = sage0.eval("import doctest, sys, os, multiprocessing, subprocess")sage: _ = sage0.eval("from sage.doctest.parsing import SageOutputChecker")sage: _ = sage0.eval("import sage.doctest.forker as sdf")
sage: _ = sage0.eval("from sage.doctest.control import DocTestDefaults")
sage: _ = sage0.eval("DD = DocTestDefaults(debug=True)")sage: _ = sage0.eval("ex1 = doctest.Example( a = 17 ,)")sage: _ = sage0.eval("ex2 = doctest.Example( 2*a , 1 )")sage: _ = sage0.eval("DT = doctest.DocTest([ex1,ex2], globals(), doubling ,␣˓→None, 0, None)")sage: _ = sage0.eval("DTR = sdf.SageDocTestRunner(SageOutputChecker(),␣
˓→verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_
˓→WHITESPACE|doctest.ELLIPSIS)")sage: print(sage0.eval("sdf.init_sage(); DTR.run(DT, clear_globs=False)")) #␣˓→indirect doctest**********************************************************************
Line 1, in doubling
Failed example:
2*a
Expected:(continues on next page)

52

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

1
Got:
34
**********************************************************************
Previously executed commands:
sage: sage0._expect.expect( sage: )# sage0 just mis-identified the output␣
˓→as prompt, synchronize0
sage: sage0.eval("a")...17sage: sage0.eval("quit")
Returning to doctests...TestResults(failed=1, attempted=2)>>> from sage.all import *>>> sage0.quit()>>> _ = sage0.eval("import doctest, sys, os, multiprocessing, subprocess")>>> _ = sage0.eval("from sage.doctest.parsing import SageOutputChecker")>>> _ = sage0.eval("import sage.doctest.forker as sdf")>>> _ = sage0.eval("from sage.doctest.control import DocTestDefaults")>>> _ = sage0.eval("DD = DocTestDefaults(debug=True)")>>> _ = sage0.eval("ex1 = doctest.Example( a = 17 ,)")>>> _ = sage0.eval("ex2 = doctest.Example( 2*a , 1 )")>>> _ = sage0.eval("DT = doctest.DocTest([ex1,ex2], globals(), doubling ,␣˓→None, 0, None)")>>> _ = sage0.eval("DTR = sdf.SageDocTestRunner(SageOutputChecker(),␣
˓→verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_
˓→WHITESPACE|doctest.ELLIPSIS)")>>> print(sage0.eval("sdf.init_sage(); DTR.run(DT, clear_globs=False)")) #␣˓→indirect doctest**********************************************************************
Line 1, in doubling
Failed example:
2*a
Expected:1
Got:34**********************************************************************
Previously executed commands:>>> sage0._expect.expect( sage: )# sage0 just mis-identified the output␣
˓→as prompt, synchronize0>>> sage0.eval("a")...17>>> sage0.eval("quit")
Returning to doctests...TestResults(failed=1, attempted=2)

report_overtime(out, test, example, got, check_duration)
Called when the warn_long option flag is set and a doctest runs longer than the specified time.
INPUT:• out – a function for printing• test – a doctest.DocTest instance• example – a doctest.Example instance in test
• got – a string, the result of running example
53

Sage's Doctesting Framework, Release 10.4

• check_duration – number (default: 0) time spent for checking the test output
OUTPUT:• prints a report to out
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.misc.timing import walltime
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = sage.doctest.forker.__file__
sage: FDS = FileDocTestSource(filename, DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: ex = doctests[0].examples[0]sage: ex.walltime = 1.23r
sage: DTR.report_overtime(sys.stdout.write, doctests[0], ex, BAD ANSWER\n ,␣
˓→check_duration=2.34r)**********************************************************************
File ".../sage/doctest/forker.py", line 12, in sage.doctest.forker
Warning, slow doctest:
doctest_var = 42; doctest_var^2
Test ran for 1.23 s, check ran for 2.34 s>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> from sage.misc.timing import walltime>>> import doctest, sys, os>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> filename = sage.doctest.forker.__file__>>> FDS = FileDocTestSource(filename, DD)>>> doctests, extras = FDS.create_doctests(globals())>>> ex = doctests[Integer(0)].examples[Integer(0)]>>> ex.walltime = 1.23>>> DTR.report_overtime(sys.stdout.write, doctests[Integer(0)], ex, BAD␣˓→ANSWER\n , check_duration=2.34)**********************************************************************
File ".../sage/doctest/forker.py", line 12, in sage.doctest.forker
Warning, slow doctest:
doctest_var = 42; doctest_var^2
Test ran for 1.23 s, check ran for 2.34 sreport_start(out, test, example)
Called when an example starts.
INPUT:• out – a function for printing• test – a doctest.DocTest instance• example – a doctest.Example instance in test
54

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

OUTPUT:• prints a report to out
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = sage.doctest.forker.__file__
sage: FDS = FileDocTestSource(filename, DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: ex = doctests[0].examples[0]sage: DTR.report_start(sys.stdout.write, doctests[0], ex)Trying (line 12):doctest_var = 42; doctest_var^2
Expecting:1764>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> import doctest, sys, os>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> filename = sage.doctest.forker.__file__>>> FDS = FileDocTestSource(filename, DD)>>> doctests, extras = FDS.create_doctests(globals())>>> ex = doctests[Integer(0)].examples[Integer(0)]>>> DTR.report_start(sys.stdout.write, doctests[Integer(0)], ex)Trying (line 12):doctest_var = 42; doctest_var^2
Expecting:1764

report_success(out, test, example, got, check_duration)
Called when an example succeeds.
INPUT:• out – a function for printing• test – a doctest.DocTest instance• example – a doctest.Example instance in test
• got – a string, the result of running example• check_duration – number (default: 0) time spent for checking the test output
OUTPUT:• prints a report to out• if in debugging mode, starts an IPython prompt at the point of the failure
EXAMPLES:

55

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.misc.timing import walltime
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = sage.doctest.forker.__file__
sage: FDS = FileDocTestSource(filename, DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: ex = doctests[0].examples[0]sage: ex.walltime = 0.0r
sage: DTR.report_success(sys.stdout.write, doctests[0], ex, 1764 )ok [0.00 s]>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> from sage.misc.timing import walltime>>> import doctest, sys, os>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> filename = sage.doctest.forker.__file__>>> FDS = FileDocTestSource(filename, DD)>>> doctests, extras = FDS.create_doctests(globals())>>> ex = doctests[Integer(0)].examples[Integer(0)]>>> ex.walltime = 0.0>>> DTR.report_success(sys.stdout.write, doctests[Integer(0)], ex, 1764 )ok [0.00 s]report_unexpected_exception(out, test, example, exc_info)
Called when a doctest raises an exception that’s not matched by the expected output.
If debugging has been turned on, starts an interactive debugger.
INPUT:• out – a function for printing• test – a doctest.DocTest instance• example – a doctest.Example instance in test
• exc_info – the result of sys.exc_info()
OUTPUT:• prints a report to out• if in debugging mode, starts PDB with the given traceback
EXAMPLES:
sage: from sage.interfaces.sage0 import sage0
sage: sage0.quit()
sage: _ = sage0.eval("import doctest, sys, os, multiprocessing, subprocess")sage: _ = sage0.eval("from sage.doctest.parsing import SageOutputChecker")(continues on next page)

56

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: _ = sage0.eval("import sage.doctest.forker as sdf")
sage: _ = sage0.eval("from sage.doctest.control import DocTestDefaults")
sage: _ = sage0.eval("DD = DocTestDefaults(debug=True)")sage: _ = sage0.eval("ex = doctest.Example( E = EllipticCurve([0,0]); E , A␣
˓→singular Elliptic Curve )")sage: _ = sage0.eval("DT = doctest.DocTest([ex], globals(), singular_curve ,␣˓→None, 0, None)")sage: _ = sage0.eval("DTR = sdf.SageDocTestRunner(SageOutputChecker(),␣
˓→verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_
˓→WHITESPACE|doctest.ELLIPSIS)")sage: old_prompt = sage0._prompt
sage: sage0._prompt = r"\(Pdb\) "sage: sage0.eval("DTR.run(DT, clear_globs=False)") # indirect doctest... ArithmeticError(self._equation_string() + " defines a singular curve")
sage: sage0.eval("l")...if self.discriminant() == 0:...raise ArithmeticError...
sage: sage0.eval("u")...-> super().__init__(R, data, category=category)sage: sage0.eval("u")...EllipticCurve_field.__init__(self, K, ainvs)
sage: sage0.eval("p ainvs")(0, 0, 0, 0, 0)
sage: sage0._prompt = old_prompt
sage: sage0.eval("quit")
TestResults(failed=1, attempted=1)>>> from sage.all import *>>> from sage.interfaces.sage0 import sage0>>> sage0.quit()>>> _ = sage0.eval("import doctest, sys, os, multiprocessing, subprocess")>>> _ = sage0.eval("from sage.doctest.parsing import SageOutputChecker")>>> _ = sage0.eval("import sage.doctest.forker as sdf")>>> _ = sage0.eval("from sage.doctest.control import DocTestDefaults")>>> _ = sage0.eval("DD = DocTestDefaults(debug=True)")>>> _ = sage0.eval("ex = doctest.Example( E = EllipticCurve([0,0]); E , A␣
˓→singular Elliptic Curve )")>>> _ = sage0.eval("DT = doctest.DocTest([ex], globals(), singular_curve ,␣˓→None, 0, None)")>>> _ = sage0.eval("DTR = sdf.SageDocTestRunner(SageOutputChecker(),␣
˓→verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_
˓→WHITESPACE|doctest.ELLIPSIS)")>>> old_prompt = sage0._prompt>>> sage0._prompt = r"\(Pdb\) ">>> sage0.eval("DTR.run(DT, clear_globs=False)") # indirect doctest... ArithmeticError(self._equation_string() + " defines a singular curve")>>> sage0.eval("l")...if self.discriminant() == 0:...raise ArithmeticError...
>>> sage0.eval("u")...-> super().__init__(R, data, category=category)>>> sage0.eval("u")...EllipticCurve_field.__init__(self, K, ainvs)>>> sage0.eval("p ainvs")(0, 0, 0, 0, 0)>>> sage0._prompt = old_prompt>>> sage0.eval("quit")
TestResults(failed=1, attempted=1)

run(test, compileflags=0, out=None, clear_globs=True)
57

Sage's Doctesting Framework, Release 10.4

Runs the examples in a given doctest.
This function replaces doctest.DocTestRunner.run since it needs to handle spoofing. It also leaves
the display hook in place.
INPUT:• test – an instance of doctest.DocTest• compileflags – int (default: 0) the set of compiler flags used to execute examples (passed in to thecompile()).
• out – a function for writing the output (defaults to sys.stdout.write()).
• clear_globs – boolean (default: True): whether to clear the namespace after running this doctest.
OUTPUT:• f – integer, the number of examples that failed• t – the number of examples tried
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD,....:
optionflags=doctest.NORMALIZE_
˓→WHITESPACE|doctest.ELLIPSIS)
sage: filename = sage.doctest.forker.__file__
sage: FDS = FileDocTestSource(filename, DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: DTR.run(doctests[0], clear_globs=False)
TestResults(failed=0, attempted=4)>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> import doctest, sys, os>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD,...
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.
˓→ELLIPSIS)>>> filename = sage.doctest.forker.__file__>>> FDS = FileDocTestSource(filename, DD)>>> doctests, extras = FDS.create_doctests(globals())>>> DTR.run(doctests[Integer(0)], clear_globs=False)
TestResults(failed=0, attempted=4)

summarize(verbose=None)
Print results of testing to self.msgfile and return number of failures and tests run.
INPUT:• verbose – whether to print lots of stuff
58

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

OUTPUT:• returns (f, t), a doctest.TestResults instance giving the number of failures and the total
number of tests run.
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: DTR._name2ft[ sage.doctest.forker ] = (1,120)
sage: results = DTR.summarize()**********************************************************************
1 item had failures:
1 of 120 in sage.doctest.forker
sage: results
TestResults(failed=1, attempted=120)>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> import doctest, sys, os>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> DTR._name2ft[ sage.doctest.forker ] = (Integer(1),Integer(120))>>> results = DTR.summarize()**********************************************************************
1 item had failures:
1 of 120 in sage.doctest.forker>>> results
TestResults(failed=1, attempted=120)

update_digests(example)
Update global and doctest digests.
Sage’s doctest runner tracks the state of doctests so that their dependencies are known. For example, in thefollowing two lines
sage: R.<x> = ZZ[]
sage: f = x^2 + 1>>> from sage.all import *>>> R = ZZ[ x ]; (x,) = R._first_ngens(1)>>> f = x**Integer(2) + Integer(1)it records that the second line depends on the first since the first INSERTS x into the global namespace andthe second line RETRIEVES x from the global namespace.
This function updates the hashes that record these dependencies.
INPUT:• example – a doctest.Example instance
EXAMPLES:

59

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os, hashlib
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = sage.doctest.forker.__file__
sage: FDS = FileDocTestSource(filename, DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: DTR.running_global_digest.hexdigest()
d41d8cd98f00b204e9800998ecf8427e
sage: DTR.running_doctest_digest = hashlib.md5()
sage: ex = doctests[0].examples[0]; ex.predecessors = Nonesage: DTR.update_digests(ex)sage: DTR.running_global_digest.hexdigest()
3cb44104292c3a3ab4da3112ce5dc35c>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> import doctest, sys, os, hashlib>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> filename = sage.doctest.forker.__file__>>> FDS = FileDocTestSource(filename, DD)>>> doctests, extras = FDS.create_doctests(globals())>>> DTR.running_global_digest.hexdigest()
d41d8cd98f00b204e9800998ecf8427e>>> DTR.running_doctest_digest = hashlib.md5()>>> ex = doctests[Integer(0)].examples[Integer(0)]; ex.predecessors = None>>> DTR.update_digests(ex)>>> DTR.running_global_digest.hexdigest()
3cb44104292c3a3ab4da3112ce5dc35c

update_results(D)When returning results we pick out the results of interest since many attributes are not pickleable.
INPUT:• D – a dictionary to update with cputime and walltime
OUTPUT:• the number of failures (or False if there is no failure attribute)
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource, DictAsObject
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = sage.doctest.forker.__file__(continues on next page)

60

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: FDS = FileDocTestSource(filename, DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: from sage.doctest.util import Timersage: T = Timer().start()sage: DTR.run(doctests[0])TestResults(failed=0, attempted=4)
sage: T.stop().annotate(DTR)
sage: D = DictAsObject({ cputime : [], walltime : [], err : None})sage: DTR.update_results(D)0sage: sorted(list(D.items()))[( cputime , [...]), ( err , None), ( failures , 0), ( tests , 4),( walltime , [...]), ( walltime_skips , 0)]>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.sources import FileDocTestSource, DictAsObject>>> from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()>>> import doctest, sys, os>>> DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_
˓→options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> filename = sage.doctest.forker.__file__>>> FDS = FileDocTestSource(filename, DD)>>> doctests, extras = FDS.create_doctests(globals())>>> from sage.doctest.util import Timer>>> T = Timer().start()>>> DTR.run(doctests[Integer(0)])
TestResults(failed=0, attempted=4)>>> T.stop().annotate(DTR)>>> D = DictAsObject({ cputime : [], walltime : [], err : None})>>> DTR.update_results(D)0>>> sorted(list(D.items()))[( cputime , [...]), ( err , None), ( failures , 0), ( tests , 4),( walltime , [...]), ( walltime_skips , 0)]class sage.doctest.forker.SageSpoofInOut(outfile=None, infile=None)
Bases: SageObject
We replace the standard doctest._SpoofOut for three reasons:• we need to divert the output of C programs that don’t print through sys.stdout,• we want the ability to recover partial output from doctest processes that segfault.
• we also redirect stdin (usually from /dev/null) during doctests.
This class defines streams self.real_stdin, self.real_stdout and self.real_stderr whichrefer to the original streams.
INPUT:• outfile – (default: tempfile.TemporaryFile()) a seekable open file object to which stdout andstderr should be redirected.
• infile – (default: open(os.devnull)) an open file object from which stdin should be redirected.
EXAMPLES:

61

Sage's Doctesting Framework, Release 10.4

sage: import subprocess, tempfile
sage: from sage.doctest.forker import SageSpoofInOut
sage: O = tempfile.TemporaryFile()
sage: S = SageSpoofInOut(O)sage: try:....:
S.start_spoofing()....:
print("hello world")....: finally:....:
S.stop_spoofing()....:sage: S.getvalue()
hello world\nsage: _ = O.seek(0)sage: S = SageSpoofInOut(outfile=sys.stdout, infile=O)
sage: try:....:
S.start_spoofing()....:_ = subprocess.check_call("cat")....: finally:....:
S.stop_spoofing()....:
hello world
sage: O.close()>>> from sage.all import *>>> import subprocess, tempfile>>> from sage.doctest.forker import SageSpoofInOut>>> O = tempfile.TemporaryFile()>>> S = SageSpoofInOut(O)>>> try:...
S.start_spoofing()...
print("hello world")... finally:...
S.stop_spoofing()....:>>> S.getvalue()
hello world\n>>> _ = O.seek(Integer(0))>>> S = SageSpoofInOut(outfile=sys.stdout, infile=O)>>> try:...
S.start_spoofing()...
_ = subprocess.check_call("cat")... finally:...
S.stop_spoofing()....:
hello world>>> O.close()getvalue()Gets the value that has been printed to outfile since the last time this function was called.
EXAMPLES:
sage: from sage.doctest.forker import SageSpoofInOut
sage: S = SageSpoofInOut()sage: try:....:
S.start_spoofing()....:
print("step 1")(continues on next page)

62

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

(continued from previous page)....: finally:....:
S.stop_spoofing()....:sage: S.getvalue()
step 1\nsage: try:....:
S.start_spoofing()....:
print("step 2")....: finally:....:
S.stop_spoofing()....:sage: S.getvalue()
step 2\n>>> from sage.all import *>>> from sage.doctest.forker import SageSpoofInOut>>> S = SageSpoofInOut()>>> try:...
S.start_spoofing()...
print("step 1")... finally:...
S.stop_spoofing()....:>>> S.getvalue()
step 1\n>>> try:...
S.start_spoofing()...
print("step 2")... finally:...
S.stop_spoofing()....:>>> S.getvalue()
step 2\nstart_spoofing()
Set stdin to read from self.infile and stdout to print to self.outfile.
EXAMPLES:
sage: import os, tempfile
sage: from sage.doctest.forker import SageSpoofInOut
sage: O = tempfile.TemporaryFile()
sage: S = SageSpoofInOut(O)sage: try:....:
S.start_spoofing()....:
print("this is not printed")....: finally:....:
S.stop_spoofing()....:sage: S.getvalue()this is not printed\nsage: _ = O.seek(0)sage: S = SageSpoofInOut(infile=O)
sage: try:....:
S.start_spoofing()....:
v = sys.stdin.read()(continues on next page)

63

Sage's Doctesting Framework, Release 10.4

(continued from previous page)....: finally:....:
S.stop_spoofing()....:sage: v
this is not printed\n>>> from sage.all import *>>> import os, tempfile>>> from sage.doctest.forker import SageSpoofInOut>>> O = tempfile.TemporaryFile()>>> S = SageSpoofInOut(O)>>> try:...
S.start_spoofing()...
print("this is not printed")... finally:...
S.stop_spoofing()....:>>> S.getvalue()this is not printed\n>>> _ = O.seek(Integer(0))>>> S = SageSpoofInOut(infile=O)>>> try:...
S.start_spoofing()...
v = sys.stdin.read()... finally:...
S.stop_spoofing()
....:
>>> v
this is not printed\nWe also catch non-Python output:
sage: try:....:
S.start_spoofing()....:
retval = os.system(
˓→necho "good"\nfi)....: finally:....:
S.stop_spoofing()....:sage: S.getvalue()
Hello there\ngood\nsage: O.close()>>> from sage.all import *>>> try:...
S.start_spoofing()...
retval = os.system(
˓→"good"\nfi)... finally:...
S.stop_spoofing()....:>>> S.getvalue()
Hello there\ngood\n>>> O.close()echo "Hello there"\nif [ $? -eq 0 ]; then\echo "Hello there"\nif [ $? -eq 0 ]; then\necho

stop_spoofing()

64

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

Reset stdin and stdout to their original values.
EXAMPLES:
sage: from sage.doctest.forker import SageSpoofInOut
sage: S = SageSpoofInOut()sage: try:....:
S.start_spoofing()....:
print("this is not printed")....: finally:....:
S.stop_spoofing()....:sage: print("this is now printed")
this is now printed>>> from sage.all import *>>> from sage.doctest.forker import SageSpoofInOut>>> S = SageSpoofInOut()>>> try:...
S.start_spoofing()...
print("this is not printed")... finally:...
S.stop_spoofing()....:>>> print("this is now printed")
this is now printedclass sage.doctest.forker.TestResults(failed, attempted)
Bases: tuple
attempted
Alias for field number 1
failedAlias for field number 0
sage.doctest.forker.dummy_handler(sig, frame)
Dummy signal handler for SIGCHLD (just to ensure the signal isn’t ignored).
sage.doctest.forker.init_sage(controller=None)
Import the Sage library.
This function is called once at the beginning of a doctest run (rather than once for each file). It imports the Sagelibrary, sets DOCTEST_MODE to True, and invalidates any interfaces.
EXAMPLES:
sage: from sage.doctest.forker import init_sage
sage: sage.doctest.DOCTEST_MODE = False
sage: init_sage()sage: sage.doctest.DOCTEST_MODE
True>>> from sage.all import *>>> from sage.doctest.forker import init_sage>>> sage.doctest.DOCTEST_MODE = False>>> init_sage()>>> sage.doctest.DOCTEST_MODE
True

65

Sage's Doctesting Framework, Release 10.4

Check that pexpect interfaces are invalidated, but still work:
sage: gap.eval("my_test_var := 42;")
42
sage: gap.eval("my_test_var;")42
sage: init_sage()sage: gap( Group((1,2,3)(4,5), (3,4)) )Group( [ (1,2,3)(4,5), (3,4) ] )sage: gap.eval("my_test_var;")Traceback (most recent call last):...
RuntimeError: Gap produced error output...
>>> from sage.all import *>>> gap.eval("my_test_var := 42;")
42>>> gap.eval("my_test_var;")42>>> init_sage()>>> gap( Group((1,2,3)(4,5), (3,4)) )Group( [ (1,2,3)(4,5), (3,4) ] )>>> gap.eval("my_test_var;")Traceback (most recent call last):...
RuntimeError: Gap produced error output...

Check that SymPy equation pretty printer is limited in doctest mode to default width (80 chars):sage: # needs sympy
sage: from sympy import sympify
sage: from sympy.printing.pretty.pretty import PrettyPrintersage: s = sympify( +x^ .join(str(i) for i in range(30)))
sage: print(PrettyPrinter(settings={ wrap_line : True}).doprint(s))
29
28
27
26
25
24
23
22
21
20
19
18
17
x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+
16
15
14
13
12
11
10
9
8
7
6
5
4
3
x
+ x
+ x
+ x
+ x
+ x
+ x
+ x + x + x + x + x + x + x + x
2+ x>>> from sage.all import *>>> # needs sympy>>> from sympy import sympify>>> from sympy.printing.pretty.pretty import PrettyPrinter>>> s = sympify( +x^ .join(str(i) for i in range(Integer(30))))>>> print(PrettyPrinter(settings={ wrap_line : True}).doprint(s))
29
28
27
26
25
24
23
22
21
20
19
18
17
x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+ x
+
<BLANKLINE>
16
15
14
13
12
11
10
9
8
7
6
5
4
3
x
+ x
+ x
+ x
+ x
+ x
+ x
+ x + x + x + x + x + x + x + x<BLANKLINE>
2
+ x

66

Chapter 3. Processes for running doctests

Sage's Doctesting Framework, Release 10.4

The displayhook sorts dictionary keys to simplify doctesting of dictionary output:
sage: { a :23, b :34, au :56,
{ a : 23, aaa : 234, au : 56,

bbf :234, aaa :234}
b : 34, bbf : 234}>>> from sage.all import *>>> { a :Integer(23), b :Integer(34), au :Integer(56),
˓→ :Integer(234)}{ a : 23, aaa : 234, au : 56, b : 34, bbf : 234}

bbf :Integer(234),

aaasage.doctest.forker.showwarning_with_traceback(message, category, filename, lineno, file=None,line=None)
Displays a warning message with a traceback.
INPUT: see warnings.showwarning().
OUTPUT: None
EXAMPLES:
sage: from sage.doctest.forker import showwarning_with_traceback
sage: showwarning_with_traceback("bad stuff", UserWarning, "myfile.py", 0)doctest:warning...
File "<doctest sage.doctest.forker.showwarning_with_traceback[1]>", line 1, in
˓→<module>
showwarning_with_traceback("bad stuff", UserWarning, "myfile.py", Integer(0)):
UserWarning: bad stuff>>> from sage.all import *>>> from sage.doctest.forker import showwarning_with_traceback>>> showwarning_with_traceback("bad stuff", UserWarning, "myfile.py", Integer(0))
doctest:warning...
File "<doctest sage.doctest.forker.showwarning_with_traceback[1]>", line 1, in
˓→<module>
showwarning_with_traceback("bad stuff", UserWarning, "myfile.py", Integer(0)):
UserWarning: bad stuff

67

Sage's Doctesting Framework, Release 10.4

68

Chapter 3. Processes for running doctests

CHAPTERFOURPARSING DOCSTRINGSThis module contains functions and classes that parse docstrings.
AUTHORS:• David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
• Jeroen Demeyer(2014-08-28) – much improved handling of tolerances using interval arithmetic (Issue #16889).
class sage.doctest.parsing.MarkedOutput
Bases: str
A subclass of string with context for whether another string matches it.
EXAMPLES:
sage: from sage.doctest.parsing import MarkedOutput
sage: s = MarkedOutput("abc")
sage: s.rel_tol
0
sage: s.update(rel_tol = .05)
abc
sage: s.rel_tol
0.0500000000000000
sage: MarkedOutput("56 µs")
56 µs>>> from sage.all import *>>> from sage.doctest.parsing import MarkedOutput>>> s = MarkedOutput("abc")>>> s.rel_tol
0>>> s.update(rel_tol = RealNumber( .05 ))abc>>> s.rel_tol
0.0500000000000000>>> MarkedOutput("56 µs")
56 µs

abs_tol = 0
random = False
rel_tol = 069

Sage's Doctesting Framework, Release 10.4

tol = 0
update(**kwds)
EXAMPLES:
sage: from sage.doctest.parsing import MarkedOutput
sage: s = MarkedOutput("0.0007401")
sage: s.update(abs_tol = .0000001)
0.0007401
sage: s.rel_tol
0
sage: s.abs_tol
1.00000000000000e-7>>> from sage.all import *>>> from sage.doctest.parsing import MarkedOutput>>> s = MarkedOutput("0.0007401")>>> s.update(abs_tol = RealNumber( .0000001 ))0.0007401>>> s.rel_tol
0>>> s.abs_tol
1.00000000000000e-7

class sage.doctest.parsing.OriginalSource(example)
Bases: object
Context swapping out the pre-parsed source with the original for better reporting.
EXAMPLES:
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults
sage: filename = sage.doctest.forker.__file__
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: doctests, extras = FDS.create_doctests(globals())
sage: ex = doctests[0].examples[0]sage: ex.sage_source
doctest_var = 42; doctest_var^2\nsage: ex.source
doctest_var = Integer(42); doctest_var**Integer(2)\nsage: from sage.doctest.parsing import OriginalSource
sage: with OriginalSource(ex):....:ex.source
doctest_var = 42; doctest_var^2\n>>> from sage.all import *>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.control import DocTestDefaults>>> filename = sage.doctest.forker.__file__>>> FDS = FileDocTestSource(filename, DocTestDefaults())>>> doctests, extras = FDS.create_doctests(globals())>>> ex = doctests[Integer(0)].examples[Integer(0)]>>> ex.sage_source
doctest_var = 42; doctest_var^2\n>>> ex.source
doctest_var = Integer(42); doctest_var**Integer(2)\n(continues on next page)

70

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> from sage.doctest.parsing import OriginalSource>>> with OriginalSource(ex):...
ex.source
doctest_var = 42; doctest_var^2\nsage.doctest.parsing.RIFtol(*args)
Create an element of the real interval field used for doctest tolerances.
It allows large numbers like 1e1000, it parses strings with spaces like RIF(" - 1 ") out of the box and it carries
a lot of precision. The latter is useful for testing libraries using arbitrary precision but not guaranteed rounding
such as PARI. We use 1044 bits of precision, which should be good to deal with tolerances on numbers computedwith 1024 bits of precision.
The interval approach also means that we do not need to worry about rounding errors and it is also very natural tosee a number with tolerance as an interval.
EXAMPLES:

sage: from sage.doctest.parsing import RIFtol
sage: RIFtol(-1, 1)
0.?
sage: RIFtol(" - 1 ")-1sage: RIFtol("1e1000")
1.
˓→0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
˓→e1000>>> from sage.all import *>>> from sage.doctest.parsing import RIFtol>>> RIFtol(-Integer(1), Integer(1))
0.?
>>> RIFtol(" - 1 ")-1>>> RIFtol("1e1000")
1.
˓→0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
˓→e1000

class sage.doctest.parsing.SageDocTestParser(optional_tags=(), long=False, *, probed_tags=(),file_optional_tags=())
Bases: DocTestParser
A version of the standard doctest parser which handles Sage’s custom options and tolerances in floating point
arithmetic.
file_optional_tags: set[str]long: bool
optional_only: bool
optional_tags: bool | set[str]optionals: dict[str, int]

71

Sage's Doctesting Framework, Release 10.4

parse(string, *args)
A Sage specialization of doctest.DocTestParser.
INPUT:• string – the string to parse.
• name – optional string giving the name identifying string, to be used in error messages.
OUTPUT:• A list consisting of strings and doctest.Example instances. There will be at least one string between
successive examples (exactly one unless long or optional tests are removed), and it will begin and end
with a string.
EXAMPLES:
sage: from sage.doctest.parsing import SageDocTestParser
sage: DTP = SageDocTestParser(( sage , magma , guava ))sage: example = Explanatory text::\n\nsage: E = magma("EllipticCurve([1,␣˓→1, 1, -10, -10])") # optional: magma\n\nLater text
sage: parsed = DTP.parse(example)
sage: parsed[0]Explanatory text::\n\nsage: parsed[1].sage_source
E = magma("EllipticCurve([1, 1, 1, -10, -10])") # optional: magma\nsage: parsed[2]\nLater text>>> from sage.all import *>>> from sage.doctest.parsing import SageDocTestParser>>> DTP = SageDocTestParser(( sage , magma , guava ))>>> example = Explanatory text::\n\nsage: E = magma("EllipticCurve([1, 1,
˓→ 1, -10, -10])") # optional: magma\n\nLater text>>> parsed = DTP.parse(example)>>> parsed[Integer(0)]Explanatory text::\n\n>>> parsed[Integer(1)].sage_source
E = magma("EllipticCurve([1, 1, 1, -10, -10])") # optional: magma\n>>> parsed[Integer(2)]\nLater text

If the doctest parser is not created to accept a given optional argument, the corresponding examples will justbe removed:
sage: DTP2 = SageDocTestParser(( sage ,))sage: parsed2 = DTP2.parse(example)
sage: parsed2[ Explanatory text::\n\n , \nLater text ]>>> from sage.all import *>>> DTP2 = SageDocTestParser(( sage ,))>>> parsed2 = DTP2.parse(example)>>> parsed2[ Explanatory text::\n\n , \nLater text ]You can mark doctests as having a particular tolerance:

72

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

sage: example2 = sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690
sage: ex = DTP.parse(example2)[1]sage: ex.sage_source
gamma(1.6) # tol 2.0e-11\nsage: ex.want
0.893515349287690\nsage: type(ex.want)<class sage.doctest.parsing.MarkedOutput >sage: ex.want.tol
2.000000000000000000...?e-11>>> from sage.all import *>>> example2 = sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690>>> ex = DTP.parse(example2)[Integer(1)]>>> ex.sage_source
gamma(1.6) # tol 2.0e-11\n>>> ex.want
0.893515349287690\n>>> type(ex.want)<class sage.doctest.parsing.MarkedOutput >>>> ex.want.tol
2.000000000000000000...?e-11

You can use continuation lines:
sage: s = "sage: for i in range(4):\n....:
sage: ex = DTP2.parse(s)[1]sage: ex.source
for i in range(Integer(4)):\n
print(i)\nprint(i)\n....:\n">>> from sage.all import *>>> s = "sage: for i in range(4):\n....:
print(i)\n....:\n">>> ex = DTP2.parse(s)[Integer(1)]>>> ex.source
for i in range(Integer(4)):\n
print(i)\nSage currently accepts backslashes as indicating that the end of the current line should be joined to the next
line. This feature allows for breaking large integers over multiple lines but is not standard for Python doctesting. It’s not guaranteed to persist:sage: n = 1234\
....:
5678
sage: print(n)12345678
sage: type(n)<class sage.rings.integer.Integer >>>> from sage.all import *>>> n = Integer(1234)
Integer(5678)>>> print(n)12345678>>> type(n)<class sage.rings.integer.Integer >It also works without the line continuation:73

Sage's Doctesting Framework, Release 10.4

sage: m = 8765\
4321
sage: print(m)87654321>>> from sage.all import *>>> m = Integer(8765)\
4321>>> print(m)87654321

Optional tags at the start of an example block persist to the end of the block (delimited by a blank line):
sage: # long time, needs sage.rings.number_fieldsage: QQbar(I)^10000
1
sage: QQbar(I)^10000 # not testedIsage: # needs sage.rings.finite_rings
sage: GF(7)Finite Field of size 7
sage: GF(10)Traceback (most recent call last):...
ValueError: the order of a finite field must be a prime power>>> from sage.all import *>>> # long time, needs sage.rings.number_field>>> QQbar(I)**Integer(10000)
1>>> QQbar(I)**Integer(10000) # not testedI>>> # needs sage.rings.finite_rings>>> GF(Integer(7))
Finite Field of size 7>>> GF(Integer(10))
Traceback (most recent call last):...
ValueError: the order of a finite field must be a prime power

Test that Issue #26575 is resolved:
sage: example3 = sage: Zp(5,4,print_mode="digits")(5)\n...00010
sage: parsed3 = DTP.parse(example3)
sage: dte = parsed3[1]sage: dte.sage_source
Zp(5,4,print_mode="digits")(5)\nsage: dte.want...00010\n>>> from sage.all import *>>> example3 = sage: Zp(5,4,print_mode="digits")(5)\n...00010>>> parsed3 = DTP.parse(example3)>>> dte = parsed3[Integer(1)](continues on next page)

74

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> dte.sage_source
Zp(5,4,print_mode="digits")(5)\n>>> dte.want...00010\nStyle warnings:
sage: def parse(test_string):....:
return [x if isinstance(x, str)....:
else (getattr(x, warnings , None), x.sage_source, x.
˓→source)....:
for x in DTP.parse(test_string)]sage: parse( sage: 1 # optional guava mango\nsage: 2 # optional guava\nsage:␣
˓→3 # optional guava\nsage: 4 # optional guava\nsage: 5 # optional guava\n\˓→nsage: 11 # optional guava )[ ,(["Consider using a block-scoped tag by inserting the line sage: # optional␣˓→- guava
just before this line to avoid repeating the tag 5 times"],
1 # optional guava mango\n ,None # virtual doctest ),,(None, 2 # optional guava\n , Integer(2) # optional guava\n ),,(None, 3 # optional guava\n , Integer(3) # optional guava\n ),,(None, 4 # optional guava\n , Integer(4) # optional guava\n ),,(None, 5 # optional guava\n , Integer(5) # optional guava\n ),\n ,(None, 11 # optional guava\n , Integer(11) # optional guava\n ),]sage: parse( sage: 1 # optional guava\nsage: 2 # optional guava mango\nsage:␣
˓→3 # optional guava\nsage: 4 # optional guava\nsage: 5 # optional guava\n )[ ,(["Consider using a block-scoped tag by inserting the line sage: # optional␣˓→- guava
just before this line to avoid repeating the tag 5 times"],
1 # optional guava\n ,Integer(1) # optional guava\n ),,,(None, 3 # optional guava\n , Integer(3) # optional guava\n ),,(None, 4 # optional guava\n , Integer(4) # optional guava\n ),,(None, 5 # optional guava\n , Integer(5) # optional guava\n ),]sage: parse( sage: # optional mango\nsage: 1 # optional guava\nsage: 2 #␣˓→optional guava mango\nsage: 3 # optional guava\nsage: 4 # optional guava\n␣
˓→sage: 5 # optional guava\n )# optional - guava mango[ ,(["Consider updating this block-scoped tag to sage: # optional - guava mango˓→to avoid repeating the tag 5 times"],# optional mango\n ,None # virtual doctest ),(continues on next page)

75

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

,
,
,
,
,
]
sage: parse( ::\n\nsage: 1 # optional guava\nsage: 2 # optional guava␣˓→mango\nsage: 3 # optional guava\n\n::\n\nsage: 4 # optional guava\n␣˓→
sage: 5 # optional guava\n )[ ::\n\n ,(None, 1 # optional guava\n , Integer(1) # optional guava\n ),,,(None, 3 # optional guava\n , Integer(3) # optional guava\n ),\n::\n\n ,(None, 4 # optional guava\n , Integer(4) # optional guava\n ),,(None, 5 # optional guava\n , Integer(5) # optional guava\n ),]>>> from sage.all import *>>> def parse(test_string):...
return [x if isinstance(x, str)...
else (getattr(x, warnings , None), x.sage_source, x.source)...
for x in DTP.parse(test_string)]>>> parse( sage: 1 # optional guava mango\nsage: 2 # optional guava\nsage: 3
˓→# optional guava\nsage: 4 # optional guava\nsage: 5 # optional guava\n\˓→nsage: 11 # optional guava )[ ,(["Consider using a block-scoped tag by inserting the line sage: # optional␣˓→- guava
just before this line to avoid repeating the tag 5 times"],
1 # optional guava mango\n ,None # virtual doctest ),,(None, 2 # optional guava\n , Integer(2) # optional guava\n ),,(None, 3 # optional guava\n , Integer(3) # optional guava\n ),,(None, 4 # optional guava\n , Integer(4) # optional guava\n ),,(None, 5 # optional guava\n , Integer(5) # optional guava\n ),\n ,(None, 11 # optional guava\n , Integer(11) # optional guava\n ),]>>> parse( sage: 1 # optional guava\nsage: 2 # optional guava mango\nsage: 3
˓→# optional guava\nsage: 4 # optional guava\nsage: 5 # optional guava\n )[ ,(["Consider using a block-scoped tag by inserting the line sage: # optional␣˓→- guava
just before this line to avoid repeating the tag 5 times"],
1 # optional guava\n ,Integer(1) # optional guava\n ),,,(continues on next page)

76

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

(continued from previous page)(None,,(None,,(None,]

3 # optional guava\n ,Integer(3) # optional guava\n ),4 # optional guava\n ,Integer(4) # optional guava\n ),5 # optional guava\n ,Integer(5) # optional guava\n ),>>> parse( sage: # optional mango\nsage: 1 # optional guava\nsage: 2 #␣˓→optional guava mango\nsage: 3 # optional guava\nsage: 4 # optional guava\n␣
˓→sage: 5 # optional guava\n )# optional - guava mango[ ,(["Consider updating this block-scoped tag to sage: # optional - guava mango˓→to avoid repeating the tag 5 times"],# optional mango\n ,None # virtual doctest ),
,
,
,
,
,
]
>>> parse( ::\n\nsage: 1 # optional guava\nsage: 2 # optional guava␣˓→mango\nsage: 3 # optional guava\n\n::\n\nsage: 4 # optional guava\n␣˓→
sage: 5 # optional guava\n )[ ::\n\n ,(None, 1 # optional guava\n , Integer(1) # optional guava\n ),,,(None, 3 # optional guava\n , Integer(3) # optional guava\n ),\n::\n\n ,(None, 4 # optional guava\n , Integer(4) # optional guava\n ),,(None, 5 # optional guava\n , Integer(5) # optional guava\n ),]probed_tags: bool | set[str]class sage.doctest.parsing.SageOutputChecker
Bases: OutputChecker
A modification of the doctest OutputChecker that can check relative and absolute tolerance of answers.
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker, MarkedOutput,␣
˓→SageDocTestParser
sage: import doctest
sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
sage: DTP = SageDocTestParser(( sage , magma , guava ))sage: OC = SageOutputChecker()sage: example2 = sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690
sage: ex = DTP.parse(example2)[1]sage: ex.sage_source
gamma(1.6) # tol 2.0e-11\nsage: ex.want
0.893515349287690\n(continues on next page)

77

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: type(ex.want)<class sage.doctest.parsing.MarkedOutput >sage: ex.want.tol
2.000000000000000000...?e-11
sage: OC.check_output(ex.want, 0.893515349287690 , optflag)True
sage: OC.check_output(ex.want, 0.8935153492877 , optflag)True
sage: OC.check_output(ex.want, 0 , optflag)
False
sage: OC.check_output(ex.want, x + 0.8935153492877 , optflag)
False>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker, MarkedOutput,␣
˓→SageDocTestParser>>> import doctest>>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS>>> DTP = SageDocTestParser(( sage , magma , guava ))>>> OC = SageOutputChecker()>>> example2 = sage: gamma(1.6) # tol 2.0e-11\n0.893515349287690>>> ex = DTP.parse(example2)[Integer(1)]>>> ex.sage_source
gamma(1.6) # tol 2.0e-11\n>>> ex.want
0.893515349287690\n>>> type(ex.want)<class sage.doctest.parsing.MarkedOutput >>>> ex.want.tol
2.000000000000000000...?e-11>>> OC.check_output(ex.want, 0.893515349287690 , optflag)
True>>> OC.check_output(ex.want, 0.8935153492877 , optflag)
True>>> OC.check_output(ex.want, 0 , optflag)
False>>> OC.check_output(ex.want, x + 0.8935153492877 , optflag)
False

add_tolerance(wantval, want)
Enlarge the real interval element wantval according to the tolerance options in want.
INPUT:• wantval – a real interval element• want – a MarkedOutput describing the tolerance
OUTPUT:• an interval element containing wantval
EXAMPLES:
sage: from sage.doctest.parsing import MarkedOutput, SageOutputChecker
sage: OC = SageOutputChecker()sage: want_tol = MarkedOutput().update(tol=0.0001)
sage: want_abs = MarkedOutput().update(abs_tol=0.0001)(continues on next page)

78

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: want_rel = MarkedOutput().update(rel_tol=0.0001)
sage: OC.add_tolerance(RIF(pi.n(64)), want_tol).endpoints()
˓→needs sage.symbolic(3.14127849432443, 3.14190681285516)
sage: OC.add_tolerance(RIF(pi.n(64)), want_abs).endpoints()
˓→needs sage.symbolic(3.14149265358979, 3.14169265358980)
sage: OC.add_tolerance(RIF(pi.n(64)), want_rel).endpoints()
˓→needs sage.symbolic(3.14127849432443, 3.14190681285516)
sage: OC.add_tolerance(RIF(1e1000), want_tol)
1.000?e1000
sage: OC.add_tolerance(RIF(1e1000), want_abs)
1.000000000000000?e1000
sage: OC.add_tolerance(RIF(1e1000), want_rel)
1.000?e1000
sage: OC.add_tolerance(0, want_tol)
0.000?
sage: OC.add_tolerance(0, want_abs)
0.000?
sage: OC.add_tolerance(0, want_rel)0>>> from sage.all import *>>> from sage.doctest.parsing import MarkedOutput, SageOutputChecker>>> OC = SageOutputChecker()>>> want_tol = MarkedOutput().update(tol=RealNumber( 0.0001 ))>>> want_abs = MarkedOutput().update(abs_tol=RealNumber( 0.0001 ))>>> want_rel = MarkedOutput().update(rel_tol=RealNumber( 0.0001 ))>>> OC.add_tolerance(RIF(pi.n(Integer(64))), want_tol).endpoints()˓→# needs sage.symbolic(3.14127849432443, 3.14190681285516)>>> OC.add_tolerance(RIF(pi.n(Integer(64))), want_abs).endpoints()˓→# needs sage.symbolic(3.14149265358979, 3.14169265358980)>>> OC.add_tolerance(RIF(pi.n(Integer(64))), want_rel).endpoints()˓→# needs sage.symbolic(3.14127849432443, 3.14190681285516)>>> OC.add_tolerance(RIF(RealNumber( 1e1000 )), want_tol)
1.000?e1000>>> OC.add_tolerance(RIF(RealNumber( 1e1000 )), want_abs)
1.000000000000000?e1000>>> OC.add_tolerance(RIF(RealNumber( 1e1000 )), want_rel)
1.000?e1000>>> OC.add_tolerance(Integer(0), want_tol)
0.000?
>>> OC.add_tolerance(Integer(0), want_abs)
0.000?
>>> OC.add_tolerance(Integer(0), want_rel)
0

#␣

#␣

#␣

␣

␣

␣

check_output(want, got, optionflags)
Checks to see if the output matches the desired output.
If want is a MarkedOutput instance, takes into account the desired tolerance.
INPUT:

79

Sage's Doctesting Framework, Release 10.4

• want – a string or MarkedOutput• got – a string• optionflags – an integer, passed down to doctest.OutputChecker
OUTPUT:• boolean, whether got matches want up to the specified tolerance.
EXAMPLES:
sage: from sage.doctest.parsing import MarkedOutput, SageOutputChecker
sage: import doctest
sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
sage: rndstr = MarkedOutput("I m wrong!").update(random=True)
sage: tentol = MarkedOutput("10.0").update(tol=.1)
sage: tenabs = MarkedOutput("10.0").update(abs_tol=.1)
sage: tenrel = MarkedOutput("10.0").update(rel_tol=.1)
sage: zerotol = MarkedOutput("0.0").update(tol=.1)
sage: zeroabs = MarkedOutput("0.0").update(abs_tol=.1)
sage: zerorel = MarkedOutput("0.0").update(rel_tol=.1)
sage: zero = "0.0"sage: nf = "9.5"
sage: ten = "10.05"
sage: eps = "-0.05"
sage: OC = SageOutputChecker()>>> from sage.all import *>>> from sage.doctest.parsing import MarkedOutput, SageOutputChecker>>> import doctest>>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS>>> rndstr = MarkedOutput("I m wrong!").update(random=True)>>> tentol = MarkedOutput("10.0").update(tol=RealNumber( .1 ))>>> tenabs = MarkedOutput("10.0").update(abs_tol=RealNumber( .1 ))>>> tenrel = MarkedOutput("10.0").update(rel_tol=RealNumber( .1 ))>>> zerotol = MarkedOutput("0.0").update(tol=RealNumber( .1 ))>>> zeroabs = MarkedOutput("0.0").update(abs_tol=RealNumber( .1 ))>>> zerorel = MarkedOutput("0.0").update(rel_tol=RealNumber( .1 ))>>> zero = "0.0">>> nf = "9.5">>> ten = "10.05">>> eps = "-0.05">>> OC = SageOutputChecker()sage: OC.check_output(rndstr,nf,optflag)True
sage: OC.check_output(tentol,nf,optflag)True
sage: OC.check_output(tentol,ten,optflag)True
sage: OC.check_output(tentol,zero,optflag)
False
sage: OC.check_output(tenabs,nf,optflag)
False
sage: OC.check_output(tenabs,ten,optflag)
True(continues on next page)

80

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: OC.check_output(tenabs,zero,optflag)
False
sage: OC.check_output(tenrel,nf,optflag)True
sage: OC.check_output(tenrel,ten,optflag)True
sage: OC.check_output(tenrel,zero,optflag)
False
sage: OC.check_output(zerotol,zero,optflag)True
sage: OC.check_output(zerotol,eps,optflag)True
sage: OC.check_output(zerotol,ten,optflag)
False
sage: OC.check_output(zeroabs,zero,optflag)True
sage: OC.check_output(zeroabs,eps,optflag)True
sage: OC.check_output(zeroabs,ten,optflag)
False
sage: OC.check_output(zerorel,zero,optflag)True
sage: OC.check_output(zerorel,eps,optflag)
False
sage: OC.check_output(zerorel,ten,optflag)
False>>> from sage.all import *>>> OC.check_output(rndstr,nf,optflag)
True>>> OC.check_output(tentol,nf,optflag)
True>>> OC.check_output(tentol,ten,optflag)
True>>> OC.check_output(tentol,zero,optflag)
False>>> OC.check_output(tenabs,nf,optflag)
False>>> OC.check_output(tenabs,ten,optflag)
True>>> OC.check_output(tenabs,zero,optflag)
False>>> OC.check_output(tenrel,nf,optflag)
True>>> OC.check_output(tenrel,ten,optflag)
True>>> OC.check_output(tenrel,zero,optflag)
False(continues on next page)81

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> OC.check_output(zerotol,zero,optflag)
True>>> OC.check_output(zerotol,eps,optflag)
True>>> OC.check_output(zerotol,ten,optflag)
False>>> OC.check_output(zeroabs,zero,optflag)
True>>> OC.check_output(zeroabs,eps,optflag)
True>>> OC.check_output(zeroabs,ten,optflag)
False>>> OC.check_output(zerorel,zero,optflag)
True>>> OC.check_output(zerorel,eps,optflag)
False>>> OC.check_output(zerorel,ten,optflag)
False

More explicit tolerance checks:
sage: _ = x # rel tol 1e10
˓→needs sage.symbolic
sage: raise RuntimeError# rel tol 1e10
Traceback (most recent call last):...
RuntimeError
sage: 1 # abs tol 2-0.5
sage: print("0.9999")# rel tol 1e-4
1.0
sage: print("1.00001")# abs tol 1e-5
1.0
sage: 0 # rel tol 1
1>>> from sage.all import *>>> _ = x # rel tol 1e10
˓→needs sage.symbolic>>> raise RuntimeError# rel tol 1e10
Traceback (most recent call last):...
RuntimeError>>> Integer(1) # abs tol 2-0.5>>> print("0.9999")# rel tol 1e-4
1.0>>> print("1.00001")# abs tol 1e-5
1.0>>> Integer(0) # rel tol 1
1

#␣

#␣

Spaces before numbers or between the sign and number are ignored:

82

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

sage: print("[ - 1, 2]")[-1,2]# abs tol 1e-10>>> from sage.all import *>>> print("[ - 1, 2]") # abs tol 1e-10[-1,2]

Tolerance on Python 3 for string results with unicode prefix:
sage: a = Cyrano ; aCyranosage: b = [ Fermat , Euler ]; b
[ Fermat ,Euler ]
sage: c = you ; cyou>>> from sage.all import *>>> a = Cyrano ; aCyrano>>> b = [ Fermat , Euler ]; b
[ Fermat ,Euler ]>>> c = you ; cyouThis illustrates that Issue #33588 is fixed:
sage: from sage.doctest.parsing import SageOutputChecker, SageDocTestParser
sage: import doctest
sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
sage: DTP = SageDocTestParser(( sage , magma , guava ))sage: OC = SageOutputChecker()sage: example = "sage: 1.3090169943749475 # tol 1e-8\n1.3090169943749475"
sage: ex = DTP.parse(example)[1]sage: OC.check_output(ex.want, 1.3090169943749475 , optflag)True
sage: OC.check_output(ex.want, ANYTHING1.3090169943749475 , optflag)
False
sage: OC.check_output(ex.want, Long-step dual simplex will be used\n1.
˓→3090169943749475 , optflag)
True>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker, SageDocTestParser>>> import doctest>>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS>>> DTP = SageDocTestParser(( sage , magma , guava ))>>> OC = SageOutputChecker()>>> example = "sage: 1.3090169943749475 # tol 1e-8\n1.3090169943749475">>> ex = DTP.parse(example)[Integer(1)]>>> OC.check_output(ex.want, 1.3090169943749475 , optflag)
True>>> OC.check_output(ex.want, ANYTHING1.3090169943749475 , optflag)
False>>> OC.check_output(ex.want, Long-step dual simplex will be used\n1.
˓→3090169943749475 , optflag)
True

83

Sage's Doctesting Framework, Release 10.4

do_fixup(want, got)
Performs few changes to the strings want and got.
For example, remove warnings to be ignored.
INPUT:• want – a string or MarkedOutput• got – a string
OUTPUT:
A tuple:• bool, True when some fixup were performed and False otherwise• string, edited wanted string• string, edited got string
Note: Currently, the code only possibly changes the string got while keeping want invariant. We keep
open the possibility of adding a regular expression which would also change the want string. This is why
want is an input and an output of the method even if currently kept invariant.
EXAMPLES:
sage: from sage.doctest.parsing import SageOutputChecker
sage: OC = SageOutputChecker()sage: OC.do_fixup( 1.3090169943749475 , 1.3090169943749475 )(False, 1.3090169943749475 , 1.3090169943749475 )sage: OC.do_fixup( 1.3090169943749475 , ANYTHING1.3090169943749475 )(False, 1.3090169943749475 , ANYTHING1.3090169943749475 )sage: OC.do_fixup( 1.3090169943749475 , Long-step dual simplex will be used\
˓→n1.3090169943749475 )(True, 1.3090169943749475 , \n1.3090169943749475 )>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker>>> OC = SageOutputChecker()>>> OC.do_fixup( 1.3090169943749475 , 1.3090169943749475 )(False, 1.3090169943749475 , 1.3090169943749475 )>>> OC.do_fixup( 1.3090169943749475 , ANYTHING1.3090169943749475 )(False, 1.3090169943749475 , ANYTHING1.3090169943749475 )>>> OC.do_fixup( 1.3090169943749475 , Long-step dual simplex will be used\n1.
˓→3090169943749475 )(True, 1.3090169943749475 , \n1.3090169943749475 )When want is an instance of class MarkedOutput:
sage: from sage.doctest.parsing import SageOutputChecker, SageDocTestParser
sage: import doctest
sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
sage: DTP = SageDocTestParser(( sage , magma , guava ))sage: OC = SageOutputChecker()sage: example = "sage: 1.3090169943749475\n1.3090169943749475"
sage: ex = DTP.parse(example)[1]sage: ex.want
1.3090169943749475\n(continues on next page)

84

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: OC.do_fixup(ex.want, 1.3090169943749475 )(False, 1.3090169943749475\n , 1.3090169943749475 )sage: OC.do_fixup(ex.want, ANYTHING1.3090169943749475 )(False, 1.3090169943749475\n , ANYTHING1.3090169943749475 )sage: OC.do_fixup(ex.want, Long-step dual simplex will be used\n1.
˓→3090169943749475 )(True, 1.3090169943749475\n , \n1.3090169943749475 )>>> from sage.all import *>>> from sage.doctest.parsing import SageOutputChecker, SageDocTestParser>>> import doctest>>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS>>> DTP = SageDocTestParser(( sage , magma , guava ))>>> OC = SageOutputChecker()>>> example = "sage: 1.3090169943749475\n1.3090169943749475">>> ex = DTP.parse(example)[Integer(1)]>>> ex.want
1.3090169943749475\n>>> OC.do_fixup(ex.want, 1.3090169943749475 )(False, 1.3090169943749475\n , 1.3090169943749475 )>>> OC.do_fixup(ex.want, ANYTHING1.3090169943749475 )(False, 1.3090169943749475\n , ANYTHING1.3090169943749475 )>>> OC.do_fixup(ex.want, Long-step dual simplex will be used\n1.
˓→3090169943749475 )(True, 1.3090169943749475\n , \n1.3090169943749475 )human_readable_escape_sequences(string)
Make ANSI escape sequences human readable.
EXAMPLES:
sage: print( This is \x1b[1mbold\x1b[0m text )This is <CSI-1m>bold<CSI-0m> text>>> from sage.all import *>>> print( This is \x1b[1mbold\x1b[0m text )This is <CSI-1m>bold<CSI-0m> text

output_difference(example, got, optionflags)
Report on the differences between the desired result and what was actually obtained.
If want is a MarkedOutput instance, takes into account the desired tolerance.
INPUT:• example – a doctest.Example instance• got – a string• optionflags – an integer, passed down to doctest.OutputChecker
OUTPUT:• a string, describing how got fails to match example.want
EXAMPLES:

85

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.parsing import MarkedOutput, SageOutputChecker
sage: import doctest
sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
sage: tentol = doctest.Example( ,MarkedOutput("10.0\n").update(tol=.1))
sage: tenabs = doctest.Example( ,MarkedOutput("10.0\n").update(abs_tol=.1))
sage: tenrel = doctest.Example( ,MarkedOutput("10.0\n").update(rel_tol=.1))
sage: zerotol = doctest.Example( ,MarkedOutput("0.0\n").update(tol=.1))
sage: zeroabs = doctest.Example( ,MarkedOutput("0.0\n").update(abs_tol=.1))
sage: zerorel = doctest.Example( ,MarkedOutput("0.0\n").update(rel_tol=.1))
sage: tlist = doctest.Example( ,MarkedOutput("[10.0, 10.0, 10.0, 10.0, 10.0,␣
˓→10.0]\n").update(abs_tol=0.987))
sage: zero = "0.0"sage: nf = "9.5"
sage: ten = "10.05"
sage: eps = "-0.05"
sage: L = "[9.9, 8.7, 10.3, 11.2, 10.8, 10.0]"
sage: OC = SageOutputChecker()>>> from sage.all import *>>> from sage.doctest.parsing import MarkedOutput, SageOutputChecker>>> import doctest>>> optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS>>> tentol = doctest.Example( ,MarkedOutput("10.0\n").update(tol=RealNumber(
˓→ .1 )))>>> tenabs = doctest.Example( ,MarkedOutput("10.0\n").update(abs_
˓→tol=RealNumber( .1 )))>>> tenrel = doctest.Example( ,MarkedOutput("10.0\n").update(rel_˓→tol=RealNumber( .1 )))>>> zerotol = doctest.Example( ,MarkedOutput("0.0\n").update(tol=RealNumber(
˓→ .1 )))>>> zeroabs = doctest.Example( ,MarkedOutput("0.0\n").update(abs_
˓→tol=RealNumber( .1 )))>>> zerorel = doctest.Example( ,MarkedOutput("0.0\n").update(rel_˓→tol=RealNumber( .1 )))>>> tlist = doctest.Example( ,MarkedOutput("[10.0, 10.0, 10.0, 10.0, 10.0,␣
˓→10.0]\n").update(abs_tol=RealNumber( 0.987 )))>>> zero = "0.0">>> nf = "9.5">>> ten = "10.05">>> eps = "-0.05">>> L = "[9.9, 8.7, 10.3, 11.2, 10.8, 10.0]">>> OC = SageOutputChecker()sage: print(OC.output_difference(tenabs,nf,optflag))
Expected:
10.0
Got:
9.5
Tolerance exceeded:10.0 vs 9.5, tolerance 5e-1 > 1e-1
sage: print(OC.output_difference(tentol,zero,optflag))
Expected:
10.0
Got:
0.0
Tolerance exceeded:(continues on next page)

86

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

(continued from previous page)10.0 vs 0.0, tolerance 1e0 > 1e-1
sage: print(OC.output_difference(tentol,eps,optflag))
Expected:
10.0
Got:
-0.05
Tolerance exceeded:10.0 vs -0.05, tolerance 2e0 > 1e-1
sage: print(OC.output_difference(tlist,L,optflag))
Expected:[10.0, 10.0, 10.0, 10.0, 10.0, 10.0]
Got:[9.9, 8.7, 10.3, 11.2, 10.8, 10.0]
Tolerance exceeded in 2 of 6:
10.0 vs 8.7, tolerance 2e0 > 9.87e-1
10.0 vs 11.2, tolerance 2e0 > 9.87e-1>>> from sage.all import *>>> print(OC.output_difference(tenabs,nf,optflag))
Expected:
10.0
Got:
9.5
Tolerance exceeded:10.0 vs 9.5, tolerance 5e-1 > 1e-1>>> print(OC.output_difference(tentol,zero,optflag))
Expected:
10.0
Got:
0.0
Tolerance exceeded:10.0 vs 0.0, tolerance 1e0 > 1e-1>>> print(OC.output_difference(tentol,eps,optflag))
Expected:
10.0
Got:
-0.05
Tolerance exceeded:10.0 vs -0.05, tolerance 2e0 > 1e-1>>> print(OC.output_difference(tlist,L,optflag))
Expected:[10.0, 10.0, 10.0, 10.0, 10.0, 10.0]
Got:[9.9, 8.7, 10.3, 11.2, 10.8, 10.0]
Tolerance exceeded in 2 of 6:
10.0 vs 8.7, tolerance 2e0 > 9.87e-1
10.0 vs 11.2, tolerance 2e0 > 9.87e-1

sage.doctest.parsing.get_source(example)
Return the source with the leading ‘sage: ‘ stripped off.
EXAMPLES:
87

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.parsing import get_source
sage: from sage.doctest.sources import DictAsObject
sage: example = DictAsObject({})
sage: example.sage_source = "2 + 2"
sage: example.source = "sage: 2 + 2"
sage: get_source(example)
2 + 2
sage: example = DictAsObject({})
sage: example.source = "3 + 3"
sage: get_source(example)
3 + 3>>> from sage.all import *>>> from sage.doctest.parsing import get_source>>> from sage.doctest.sources import DictAsObject>>> example = DictAsObject({})>>> example.sage_source = "2 + 2">>> example.source = "sage: 2 + 2">>> get_source(example)
2 + 2>>> example = DictAsObject({})>>> example.source = "3 + 3">>> get_source(example)
3 + 3

sage.doctest.parsing.make_marked_output(s, D)
Auxiliary function for pickling.
EXAMPLES:
sage: from sage.doctest.parsing import make_marked_output
sage: s = make_marked_output("0.0007401", { abs_tol :.0000001})
sage: s0.0007401
sage: s.abs_tol
1.00000000000000e-7>>> from sage.all import *>>> from sage.doctest.parsing import make_marked_output>>> s = make_marked_output("0.0007401", { abs_tol :RealNumber( .0000001 )})>>> s
0.0007401>>> s.abs_tol
1.00000000000000e-7

sage.doctest.parsing.parse_file_optional_tags(lines)
Scan the first few lines for file-level doctest directives.
INPUT:• lines – iterable of pairs (lineno, line).
OUTPUT:a dictionary whose keys are strings (tags); see parse_optional_tags()
EXAMPLES:

88

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.parsing import parse_file_optional_tags
sage: filename = tmp_filename(ext=".pyx")
sage: with open(filename, "r") as f:....:parse_file_optional_tags(enumerate(f)){}
sage: with open(filename, "w") as f:....:_ = f.write("# nodoctest")
sage: with open(filename, "r") as f:....:parse_file_optional_tags(enumerate(f)){ not tested : None}sage: with open(filename, "w") as f:....:_ = f.write("# sage.doctest: "# broken in two source lines to avoid␣˓→the pattern....:"optional - xyz")# of relint (multiline_doctest_
˓→comment)
sage: with open(filename, "r") as f:....:parse_file_optional_tags(enumerate(f)){ xyz : None}>>> from sage.all import *>>> from sage.doctest.parsing import parse_file_optional_tags>>> filename = tmp_filename(ext=".pyx")>>> with open(filename, "r") as f:...
parse_file_optional_tags(enumerate(f)){}>>> with open(filename, "w") as f:...
_ = f.write("# nodoctest")>>> with open(filename, "r") as f:...
parse_file_optional_tags(enumerate(f)){ not tested : None}>>> with open(filename, "w") as f:...
_ = f.write("# sage.doctest: "# broken in two source lines to avoid␣˓→the pattern...
"optional - xyz")# of relint (multiline_doctest_comment)>>> with open(filename, "r") as f:...
parse_file_optional_tags(enumerate(f)){ xyz : None}sage.doctest.parsing.parse_optional_tags(string: str) → dict[str, str | None]sage.doctest.parsing.parse_optional_tags(string: str, *, return_string_sans_tags: Literal[True]) →tuple[dict[str, str | None], str, bool]
Return a dictionary whose keys are optional tags from the following set that occur in a comment on the first line of
the input string.
•

long time•

not implemented•

not tested•

known bug (possible values are None, linux and macos)

•

py2

•

optional -- FEATURE... or needs FEATURE... – the dictionary will just have the key
FEATUREThe values, if non-None, are strings with optional explanations for a tag, which may appear in parentheses after
the tag in string.
89

Sage's Doctesting Framework, Release 10.4

INPUT:• string – a string• return_string_sans_tags – (boolean, default False); whether to additionally return string withthe optional tags removed but other comments kept and a boolean is_persistent
EXAMPLES:
sage: from sage.doctest.parsing import parse_optional_tags
sage: parse_optional_tags("sage: magma( 2 + 2 )# optional: magma"){ magma : None}sage: parse_optional_tags("sage: #optional -- mypkg"){ mypkg : None}sage: parse_optional_tags("sage: print(1) # parentheses are optional here"){}
sage: parse_optional_tags("sage: print(1) # optional"){}
sage: sorted(list(parse_optional_tags("sage: #optional -- foo bar, baz")))[ bar , foo ]sage: parse_optional_tags("sage: #optional -- foo.bar, baz"){ foo.bar : None}sage: parse_optional_tags("sage: #needs foo.bar, baz"){ foo.bar : None}sage: sorted(list(parse_optional_tags("
sage: factor(10^(10^10) + 1) # LoNg␣
˓→TiME, NoT TeSTED; OptioNAL -- P4cka9e")))[ long time , not tested , p4cka9e ]sage: parse_optional_tags("
sage: raise RuntimeError # known bug"){ bug : None}sage: sorted(list(parse_optional_tags("
sage: determine_meaning_of_life() #␣˓→long time, not implemented")))[ long time , not implemented ]>>> from sage.all import *>>> from sage.doctest.parsing import parse_optional_tags>>> parse_optional_tags("sage: magma( 2 + 2 )# optional: magma"){ magma : None}>>> parse_optional_tags("sage: #optional -- mypkg"){ mypkg : None}>>> parse_optional_tags("sage: print(1) # parentheses are optional here"){}>>> parse_optional_tags("sage: print(1) # optional"){}>>> sorted(list(parse_optional_tags("sage: #optional -- foo bar, baz")))[ bar , foo ]>>> parse_optional_tags("sage: #optional -- foo.bar, baz"){ foo.bar : None}>>> parse_optional_tags("sage: #needs foo.bar, baz"){ foo.bar : None}>>> sorted(list(parse_optional_tags("
sage: factor(10^(10^10) + 1) # LoNg TiME,
˓→ NoT TeSTED; OptioNAL -- P4cka9e")))[ long time , not tested , p4cka9e ]>>> parse_optional_tags("
sage: raise RuntimeError # known bug"){ bug : None}>>> sorted(list(parse_optional_tags("
sage: determine_meaning_of_life() # long␣˓→time, not implemented")))[ long time , not implemented ]We don’t parse inside strings:

90

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

sage: parse_optional_tags("{}
sage: parse_optional_tags("{ not tested : None}>>> from sage.all import *>>> parse_optional_tags("{}>>> parse_optional_tags("{ not tested : None}sage: print(# long time )")sage: print(# long time )sage: print(# long time )")sage: print(# long time )# not tested")# not tested")

UTF-8 works:
sage: parse_optional_tags(" ěščřžýáíéďĎ "){}>>> from sage.all import *>>> parse_optional_tags(" ěščřžýáíéďĎ "){}Tags with parenthesized explanations:sage: parse_optional_tags("
sage: 1 + 1 # long time (1 year, 2 months??),␣
˓→optional - bliss (because)"){ bliss : because , long time : 1 year, 2 months?? }>>> from sage.all import *>>> parse_optional_tags("
sage: 1 + 1 # long time (1 year, 2 months??),␣
˓→optional - bliss (because)"){ bliss : because , long time : 1 year, 2 months?? }With return_string_sans_tags=True:
sage: parse_optional_tags("sage: print(1) # very important 1 # optional - foo",....:
return_string_sans_tags=True)({ foo : None}, sage: print(1) # very important 1, False)sage: parse_optional_tags("sage: print(# very important too # optional - foo\
˓→n....:
2)",....:
return_string_sans_tags=True)({ foo : None}, sage: print(# very important too \n....:
2) , False)
sage: parse_optional_tags("sage: #this is persistent #needs scipy",....:
return_string_sans_tags=True)({ scipy : None}, sage: #this is persistent , True)
sage: parse_optional_tags("sage: #this is not #needs scipy\n....: import scipy",....:
return_string_sans_tags=True)({ scipy : None}, sage: #this is not \n....: import scipy , False)>>> from sage.all import *>>> parse_optional_tags("sage: print(1) # very important 1 # optional - foo",...
return_string_sans_tags=True)({ foo : None}, sage: print(1) # very important 1, False)>>> parse_optional_tags("sage: print(# very important too # optional - foo\n.
˓→...:
2)",...
return_string_sans_tags=True)({ foo : None}, sage: print(# very important too \n....:
2) , False)>>> parse_optional_tags("sage: #this is persistent #needs scipy",(continues on next page)

91

Sage's Doctesting Framework, Release 10.4

(continued from previous page)...
return_string_sans_tags=True)({ scipy : None}, sage: #this is persistent , True)>>> parse_optional_tags("sage: #this is not #needs scipy\n....: import scipy",...
return_string_sans_tags=True)({ scipy : None}, sage: #this is not \n....: import scipy , False)

sage.doctest.parsing.parse_tolerance(source, want)
Return a version of want marked up with the tolerance tags specified in source.
INPUT:• source – a string, the source of a doctest• want – a string, the desired output of the doctest
OUTPUT:
want if there are no tolerance tags specified; a MarkedOutput version otherwise.
EXAMPLES:
sage: from sage.doctest.parsing import parse_tolerance
sage: marked = parse_tolerance("sage: s.update(abs_tol = .0000001)", "")sage: type(marked)<class str >sage: marked = parse_tolerance("sage: s.update(tol = 0.1); s.rel_tol # abs tol
˓→ 0.01 ", "")sage: marked.tol
0
sage: marked.rel_tol
0
sage: marked.abs_tol
0.010000000000000000000...?
>>> from sage.all import *>>> from sage.doctest.parsing import parse_tolerance>>> marked = parse_tolerance("sage: s.update(abs_tol = .0000001)", "")>>> type(marked)<class str >>>> marked = parse_tolerance("sage: s.update(tol = 0.1); s.rel_tol # abs tol
˓→0.01 ", "")>>> marked.tol
0>>> marked.rel_tol
0>>> marked.abs_tol
0.010000000000000000000...?

␣␣

sage.doctest.parsing.pre_hash(s)Prepends a string with its length.
EXAMPLES:
sage: from sage.doctest.parsing import pre_hashsage: pre_hash("abc")
3:abc

92

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4>>> from sage.all import *>>> from sage.doctest.parsing import pre_hash>>> pre_hash("abc")
3:abc

sage.doctest.parsing.reduce_hex(fingerprints)
Return a symmetric function of the arguments as hex strings.
The arguments should be 32 character strings consisting of hex digits: 0-9 and a-f.
EXAMPLES:
sage: from sage.doctest.parsing import reduce_hex
sage: reduce_hex(["abc", "12399aedf"])
0000000000000000000000012399a463
sage: reduce_hex(["12399aedf","abc"])
0000000000000000000000012399a463>>> from sage.all import *>>> from sage.doctest.parsing import reduce_hex>>> reduce_hex(["abc", "12399aedf"])
0000000000000000000000012399a463>>> reduce_hex(["12399aedf","abc"])
0000000000000000000000012399a463

sage.doctest.parsing.unparse_optional_tags(tags, prefix='# ')Return a comment string that sets tags.
INPUT:• tags – dict or iterable of tags, as output by parse_optional_tags()• prefix – to be put before a nonempty string
EXAMPLES:
sage: from sage.doctest.parsing import unparse_optional_tags
sage: unparse_optional_tags({})
sage: unparse_optional_tags({ magma : None})# optional - magma
sage: unparse_optional_tags({ fictional_optional : None,....:sage.rings.number_field : None,....:scipy : just because ,....:bliss : None})# optional - bliss fictional_optional, needs scipy (just because) sage.rings.
˓→number_fieldsage: unparse_optional_tags([ long time , not tested , p4cka9e ], prefix= )long time, not tested, optional - p4cka9e>>> from sage.all import *>>> from sage.doctest.parsing import unparse_optional_tags>>> unparse_optional_tags({})>>> unparse_optional_tags({ magma : None})# optional - magma>>> unparse_optional_tags({ fictional_optional : None,...
sage.rings.number_field : None,(continues on next page)

93

Sage's Doctesting Framework, Release 10.4

(continued from previous page)...
scipy : just because ,...
bliss : None})# optional - bliss fictional_optional, needs scipy (just because) sage.rings.
˓→number_field>>> unparse_optional_tags([ long time , not tested , p4cka9e ], prefix= )long time, not tested, optional - p4cka9esage.doctest.parsing.update_optional_tags(line, tags, add_tags, remove_tags, force_rewrite=None)
Return the doctest line with tags changed.
EXAMPLES:
sage: from sage.doctest.parsing import update_optional_tags, optional_tag_columns,
˓→ standard_tag_columns
sage: ruler =sage: for column in optional_tag_columns:....:
ruler +=* (column - len(ruler)) + V
sage: for column in standard_tag_columns:....:
ruler +=* (column - len(ruler)) + v
sage: def print_with_ruler(lines):....:
print( | + ruler)....:
for line in lines:....:
print( | + line)
sage: print_with_ruler([ # the tags are obscured in the source file to avoid␣˓→relint warnings....:
update_optional_tags(
sage: something() # opt
ional - latte_int ,....:remove_tags=[ latte_int , wasnt_even_there ]),....:update_optional_tags(
sage: nothing_to_be_seen_here() ,....:tags=[ scipy , long time ]),....:update_optional_tags(
sage: nothing_to_be_seen_here(honestly=True) ,....:add_tags=[ scipy , long time ]),....:update_optional_tags(
sage: nothing_to_be_seen_here(honestly=True,␣
˓→very=True) ,....:add_tags=[ scipy , long time ]),....:update_optional_tags(
sage: no_there_is_absolutely_nothing_to_be_
˓→seen_here_i_am_serious()#opt
ional:bliss ,....:add_tags=[ scipy , long time ]),....:update_optional_tags(
sage: ntbsh() # abbrv for above#opt
˓→ ional:bliss ,....:add_tags={ scipy : None, long time : 30s on the␣˓→highest setting }),....:update_optional_tags(
sage: no_there_is_absolutely_nothing_to_be_
˓→seen_here_i_am_serious()# really, you can trust me here ,....:add_tags=[ scipy ]),....: ])
|
V
V
V
V
␣
˓→V
V
v
v
v
␣
˓→v
|
sage: something()|sage: nothing_to_be_seen_here()# long time␣˓→# needs scipy|
sage: nothing_to_be_seen_here(honestly=True)# long time␣˓→# needs scipy|
sage: nothing_to_be_seen_here(honestly=True, very=True)# long time␣˓→# needs scipy|
sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()
˓→# long time, optional - bliss, needs scipy(continues on next page)

94

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

(continued from previous page)|

sage: ntbsh() # abbrv for above# long time (30s on the highest␣˓→setting), optional - bliss, needs scipy|
sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious() #␣˓→really, you can trust me here
# needs scipy>>> from sage.all import *>>> from sage.doctest.parsing import update_optional_tags, optional_tag_columns,␣
˓→standard_tag_columns>>> ruler =>>> for column in optional_tag_columns:...
ruler +=* (column - len(ruler)) + V>>> for column in standard_tag_columns:...
ruler +=* (column - len(ruler)) + v>>> def print_with_ruler(lines):...
print( | + ruler)...
for line in lines:...
print( | + line)>>> print_with_ruler([ # the tags are obscured in the source file to avoid␣˓→relint warnings...
update_optional_tags(
sage: something() # opt
ional - latte_int ,...
remove_tags=[ latte_int , wasnt_even_there ]),...
update_optional_tags(
sage: nothing_to_be_seen_here() ,...
tags=[ scipy , long time ]),...
update_optional_tags(
sage: nothing_to_be_seen_here(honestly=True) ,...
add_tags=[ scipy , long time ]),...
update_optional_tags(
sage: nothing_to_be_seen_here(honestly=True,␣
˓→very=True) ,...
add_tags=[ scipy , long time ]),...
update_optional_tags(
sage: no_there_is_absolutely_nothing_to_be_seen_
˓→here_i_am_serious()#opt
ional:bliss ,...
add_tags=[ scipy , long time ]),...
update_optional_tags(
sage: ntbsh() # abbrv for above#opt
˓→ ional:bliss ,...
add_tags={ scipy : None, long time : 30s on the␣˓→highest setting }),...
update_optional_tags(
sage: no_there_is_absolutely_nothing_to_be_seen_
˓→here_i_am_serious()# really, you can trust me here ,...
add_tags=[ scipy ]),... ])
|
V
V
V
V
␣
˓→V
V
v
v
v
␣
˓→v
|
sage: something()|sage: nothing_to_be_seen_here()# long time␣˓→# needs scipy|
sage: nothing_to_be_seen_here(honestly=True)# long time␣˓→# needs scipy|
sage: nothing_to_be_seen_here(honestly=True, very=True)# long time␣˓→# needs scipy|
sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious()
˓→# long time, optional - bliss, needs scipy|
sage: ntbsh() # abbrv for above# long time (30s on the highest␣˓→setting), optional - bliss, needs scipy|
sage: no_there_is_absolutely_nothing_to_be_seen_here_i_am_serious() #␣˓→really, you can trust me here
# needs scipy

When no tags are changed, by default, the unchanged input is returned. We can force a rewrite; unconditionally or
95

Sage's Doctesting Framework, Release 10.4

whenever standard tags are involved. But even when forced, if comments are already aligned at one of the standardalignment columns, this alignment is kept even if we would normally realign farther to the left:
sage: print_with_ruler([....:
update_optional_tags(
sage: unforced()# opt
ional - latte_
˓→int ),....:
update_optional_tags(
sage: unforced() # opt
ional - latte_int ,....:add_tags=[ latte_int ]),....:update_optional_tags(
sage: forced()#opt
ional- latte_int ,....:force_rewrite=True),....:
update_optional_tags(
sage: forced() # opt
ional - scipy ,....:force_rewrite= standard ),....:update_optional_tags(
sage: aligned_with_below()
␣
˓→
# opt
ional - 4ti2 ,....:force_rewrite=True),....:
update_optional_tags(
sage: aligned_with_above()
␣
˓→
# opt
ional - 4ti2 ,....:force_rewrite=True),....:
update_optional_tags(
sage: also_already_aligned()
␣
˓→
# ne
˓→ eds scipy ,....:force_rewrite= standard ),....:update_optional_tags(
sage: two_columns_first_preserved()#␣˓→long time# ne
eds scipy ,....:force_rewrite= standard ),....:update_optional_tags(
sage: two_columns_first_preserved()
␣
˓→
# lo
ng time# ne
eds scipy ,....:force_rewrite= standard ),....: ])
|
V
V
V
V
␣
˓→V
V
v
v
v
␣
˓→v
|
sage: unforced()# optional - latte_int|
sage: unforced() # optional - latte_int|
sage: forced()# optional - latte_int|
sage: forced()␣˓→# needs scipy|
sage: aligned_with_below()# optional - 4ti2|
sage: aligned_with_above()# optional - 4ti2|
sage: also_already_aligned()␣˓→# needs scipy|
sage: two_columns_first_preserved()# long time␣˓→# needs scipy|
sage: two_columns_first_preserved()# long time␣˓→# needs scipy>>> from sage.all import *>>> print_with_ruler([...
update_optional_tags(
sage: unforced()# opt
ional - latte_int
˓→ ),
...
update_optional_tags(
sage: unforced() # opt
ional - latte_int ,...
add_tags=[ latte_int ]),...
update_optional_tags(
sage: forced()#opt
ional- latte_int ,...
force_rewrite=True),...
update_optional_tags(
sage: forced() # opt
ional - scipy ,...
force_rewrite= standard ),...
update_optional_tags(
sage: aligned_with_below()
␣
˓→
# opt
ional - 4ti2 ,(continues on next page)

96

Chapter 4. Parsing docstrings

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

...
...
˓→

...
...

force_rewrite=True),
update_optional_tags(
sage: aligned_with_above()# opt
ional - 4ti2 ,force_rewrite=True),
update_optional_tags(
sage: also_already_aligned()

˓→

␣

␣
# ne

eds␣

˓→scipy ,...
force_rewrite= standard ),...
update_optional_tags(
sage: two_columns_first_preserved()# lo
˓→ng time# ne
eds scipy ,...
force_rewrite= standard ),...
update_optional_tags(
sage: two_columns_first_preserved()
␣
˓→
# lo
ng time# ne
eds scipy ,...
force_rewrite= standard ),... ])
|
V
V
V
V
␣
˓→V
V
v
v
v
␣
˓→v
|
sage: unforced()# optional - latte_int|
sage: unforced() # optional - latte_int|
sage: forced()# optional - latte_int|
sage: forced()␣˓→# needs scipy|
sage: aligned_with_below()# optional - 4ti2|
sage: aligned_with_above()# optional - 4ti2|
sage: also_already_aligned()␣˓→# needs scipy|
sage: two_columns_first_preserved()# long time␣˓→# needs scipy|
sage: two_columns_first_preserved()# long time␣˓→# needs scipy

Rewriting a persistent (block-scoped) tag:sage: print_with_ruler([....:
update_optional_tags(
sage:
#opt
ional:magma sage.symbolic ,....:force_rewrite=True),....: ])
|
V
V
V
V
␣
˓→V
V
v
v
v
␣
˓→v
|
sage: # optional - magma, needs sage.symbolic>>> from sage.all import *>>> print_with_ruler([...
update_optional_tags(
sage:
#opt
ional:magma sage.symbolic ,...
force_rewrite=True),... ])
|
V
V
V
V
˓→V
V
v
v
v
˓→v
|
sage: # optional - magma, needs sage.symbolic

␣
␣

97

Sage's Doctesting Framework, Release 10.4

98

Chapter 4. Parsing docstrings

CHAPTERFIVEREPORTING DOCTEST RESULTS

This module determines how doctest results are reported to the user.
It also computes the exit status in the error_status attribute of DocTestReporter. This is a bitwise OR of thefollowing bits:• 1: Doctest failure• 2: Bad command line syntax or invalid options• 4: Test timed out• 8: Test exited with non-zero status• 16: Test crashed with a signal (e.g. segmentation fault)• 32: TAB character found• 64: Internal error in the doctesting framework• 128: Testing interrupted, not all tests run• 256: Doctest contains explicit source line number
AUTHORS:• David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
class sage.doctest.reporting.DocTestReporter(controller)
Bases: SageObject
This class reports to the users on the results of doctests.
finalize()
Print out the postscript that summarizes the doctests that were run.
EXAMPLES:First we have to set up a bunch of stuff:
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource, DictAsObject
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.util import Timersage: import doctest
sage: filename = sage.doctest.reporting.__file__
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename, DD)(continues on next page)

99

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: DC = DocTestController(DD, [filename])
sage: DTR = DocTestReporter(DC)>>> from sage.all import *>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource, DictAsObject>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.util import Timer>>> import doctest>>> filename = sage.doctest.reporting.__file__>>> DD = DocTestDefaults()>>> FDS = FileDocTestSource(filename, DD)>>> DC = DocTestController(DD, [filename])>>> DTR = DocTestReporter(DC)Now we pretend to run some doctests:
sage: DTR.report(FDS, True, 0, None, "Output so far...", pid=1234)
Timed out**********************************************************************
Tests run before process (pid=1234) timed out:
Output so far...
**********************************************************************
sage: DTR.report(FDS, False, 3, None, "Output before bad exit")
Bad exit: 3**********************************************************************
Tests run before process failed:Output before bad exit**********************************************************************
sage: doctests, extras = FDS.create_doctests(globals())
sage: runner = SageDocTestRunner(....:
SageOutputChecker(), verbose=False, sage_options=DD,....:
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: t = Timer().start().stop()
sage: t.annotate(runner)
sage: DC.timer = t
sage: D = DictAsObject({ err :None})sage: runner.update_results(D)0sage: DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D),....:"Good tests")[... tests, ... s]sage: runner.failures = 1
sage: runner.update_results(D)1sage: DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D),....:"Doctest output including the failure...")[... tests, 1 failure, ... s]>>> from sage.all import *>>> DTR.report(FDS, True, Integer(0), None, "Output so far...",␣
˓→pid=Integer(1234))
Timed out**********************************************************************(continues on next page)

100

Chapter 5. Reporting doctest results

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

Tests run before process (pid=1234) timed out:
Output so far...
**********************************************************************>>> DTR.report(FDS, False, Integer(3), None, "Output before bad exit")
Bad exit: 3**********************************************************************
Tests run before process failed:Output before bad exit**********************************************************************>>> doctests, extras = FDS.create_doctests(globals())>>> runner = SageDocTestRunner(...
SageOutputChecker(), verbose=False, sage_options=DD,...
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> t = Timer().start().stop()>>> t.annotate(runner)>>> DC.timer = t>>> D = DictAsObject({ err :None})>>> runner.update_results(D)0>>> DTR.report(FDS, False, Integer(0), (sum([len(t.examples) for t in␣˓→doctests]), D),...
"Good tests")[... tests, ... s]>>> runner.failures = Integer(1)>>> runner.update_results(D)1>>> DTR.report(FDS, False, Integer(0), (sum([len(t.examples) for t in␣˓→doctests]), D),...
"Doctest output including the failure...")[... tests, 1 failure, ... s]

Now we can show the output of finalize:
sage: DC.sources = [None] * 4 # to fool the finalize method
sage: DTR.finalize()---------------------------------------------------------------------sage -t .../sage/doctest/reporting.py # Timed out
sage -t .../sage/doctest/reporting.py # Bad exit: 3sage -t .../sage/doctest/reporting.py # 1 doctest failed---------------------------------------------------------------------Total time for all tests: 0.0 seconds
cpu time: 0.0 seconds
cumulative wall time: 0.0 seconds>>> from sage.all import *>>> DC.sources = [None] * Integer(4) # to fool the finalize method>>> DTR.finalize()---------------------------------------------------------------------sage -t .../sage/doctest/reporting.py # Timed out
sage -t .../sage/doctest/reporting.py # Bad exit: 3sage -t .../sage/doctest/reporting.py # 1 doctest failed---------------------------------------------------------------------Total time for all tests: 0.0 seconds
cpu time: 0.0 seconds
cumulative wall time: 0.0 seconds

If we interrupted doctests, then the number of files tested will not match the number of sources on the con101

Sage's Doctesting Framework, Release 10.4

troller:sage: DC.sources = [None] * 6sage: DTR.finalize()---------------------------------------------------------------------sage -t .../sage/doctest/reporting.py # Timed out
sage -t .../sage/doctest/reporting.py # Bad exit: 3sage -t .../sage/doctest/reporting.py # 1 doctest failed
Doctests interrupted: 4/6 files tested---------------------------------------------------------------------Total time for all tests: 0.0 seconds
cpu time: 0.0 seconds
cumulative wall time: 0.0 seconds>>> from sage.all import *>>> DC.sources = [None] * Integer(6)>>> DTR.finalize()<BLANKLINE>---------------------------------------------------------------------sage -t .../sage/doctest/reporting.py # Timed out
sage -t .../sage/doctest/reporting.py # Bad exit: 3sage -t .../sage/doctest/reporting.py # 1 doctest failed
Doctests interrupted: 4/6 files tested---------------------------------------------------------------------Total time for all tests: 0.0 seconds
cpu time: 0.0 seconds
cumulative wall time: 0.0 seconds

report(source, timeout, return_code, results, output, pid=None)
Report on the result of running doctests on a given source.
This doesn’t print the report_head(), which is assumed to be printed already.
INPUT:• source – a source from sage.doctest.sources• timeout – a boolean, whether doctests timed out• return_code – an int, the return code of the process running doctests on that file.
• results – (irrelevant if timeout or return_code), a tuple– ntests – the number of doctests– timings – a sage.doctest.sources.DictAsObject instance storing timing data.
• output – a string, printed if there was some kind of failure• pid – optional integer (default: None). The pid of the worker process.
EXAMPLES:
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource, DictAsObject
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.util import Timersage: import doctest(continues on next page)

102

Chapter 5. Reporting doctest results

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: filename = sage.doctest.reporting.__file__
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename, DD)
sage: DC = DocTestController(DD, [filename])
sage: DTR = DocTestReporter(DC)>>> from sage.all import *>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource, DictAsObject>>> from sage.doctest.forker import SageDocTestRunner>>> from sage.doctest.parsing import SageOutputChecker>>> from sage.doctest.util import Timer>>> import doctest>>> filename = sage.doctest.reporting.__file__>>> DD = DocTestDefaults()>>> FDS = FileDocTestSource(filename, DD)>>> DC = DocTestController(DD, [filename])>>> DTR = DocTestReporter(DC)You can report a timeout:
sage: DTR.report(FDS, True, 0, None, "Output so far...", pid=1234)
Timed out**********************************************************************
Tests run before process (pid=1234) timed out:
Output so far...
**********************************************************************
sage: DTR.stats{ sage.doctest.reporting : { failed : True,
ntests : 0,
walltime : 1000000.0}}>>> from sage.all import *>>> DTR.report(FDS, True, Integer(0), None, "Output so far...",␣
˓→pid=Integer(1234))
Timed out**********************************************************************
Tests run before process (pid=1234) timed out:
Output so far...
**********************************************************************>>> DTR.stats{ sage.doctest.reporting : { failed : True,
ntests : 0,
walltime : 1000000.0}}

Or a process that returned a bad exit code:
sage: DTR.report(FDS, False, 3, None, "Output before trouble")
Bad exit: 3**********************************************************************
Tests run before process failed:Output before trouble**********************************************************************
sage: DTR.stats{ sage.doctest.reporting : { failed : True,(continues on next page)

103

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

ntests : 0,
walltime : 1000000.0}}>>> from sage.all import *>>> DTR.report(FDS, False, Integer(3), None, "Output before trouble")
Bad exit: 3**********************************************************************
Tests run before process failed:Output before trouble**********************************************************************>>> DTR.stats{ sage.doctest.reporting : { failed : True,
ntests : 0,
walltime : 1000000.0}}

Or a process that segfaulted:
sage: from signal import SIGSEGV
sage: DTR.report(FDS, False, -SIGSEGV, None, "Output before trouble")
Killed due to segmentation fault**********************************************************************
Tests run before process failed:Output before trouble**********************************************************************
sage: DTR.stats{ sage.doctest.reporting : { failed : True,
ntests : 0,
walltime : 1000000.0}}>>> from sage.all import *>>> from signal import SIGSEGV>>> DTR.report(FDS, False, -SIGSEGV, None, "Output before trouble")
Killed due to segmentation fault**********************************************************************
Tests run before process failed:Output before trouble**********************************************************************>>> DTR.stats{ sage.doctest.reporting : { failed : True,
ntests : 0,
walltime : 1000000.0}}

Report a timeout with results and a SIGKILL:
sage: from signal import SIGKILL
sage: DTR.report(FDS, True, -SIGKILL, (1,None), "Output before trouble")
Timed out after testing finished (and interrupt failed)**********************************************************************
Tests run before process timed out:
Output before trouble**********************************************************************
sage: DTR.stats{ sage.doctest.reporting : { failed : True,
ntests : 1,
walltime : 1000000.0}}

104

Chapter 5. Reporting doctest results

Sage's Doctesting Framework, Release 10.4>>> from sage.all import *>>> from signal import SIGKILL>>> DTR.report(FDS, True, -SIGKILL, (Integer(1),None), "Output before trouble˓→")
Timed out after testing finished (and interrupt failed)**********************************************************************
Tests run before process timed out:
Output before trouble**********************************************************************>>> DTR.stats{ sage.doctest.reporting : { failed : True,
ntests : 1,
walltime : 1000000.0}}

This is an internal error since results is None:sage: DTR.report(FDS, False, 0, None, "All output")
Error in doctesting framework (bad result returned)**********************************************************************
Tests run before error:
All output**********************************************************************
sage: DTR.stats{ sage.doctest.reporting : { failed : True,
ntests : 1,
walltime : 1000000.0}}>>> from sage.all import *>>> DTR.report(FDS, False, Integer(0), None, "All output")
Error in doctesting framework (bad result returned)**********************************************************************
Tests run before error:
All output**********************************************************************>>> DTR.stats{ sage.doctest.reporting : { failed : True,
ntests : 1,
walltime : 1000000.0}}

Or tell the user that everything succeeded:sage: doctests, extras = FDS.create_doctests(globals())
sage: runner = SageDocTestRunner(....:
SageOutputChecker(), verbose=False, sage_options=DD,....:
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: Timer().start().stop().annotate(runner)
sage: D = DictAsObject({ err :None})sage: runner.update_results(D)0sage: DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D),....:"Good tests")[... tests, ... s]sage: DTR.stats{ sage.doctest.reporting : { ntests : ..., walltime : ...}}>>> from sage.all import *>>> doctests, extras = FDS.create_doctests(globals())(continues on next page)

105

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> runner = SageDocTestRunner(...
SageOutputChecker(), verbose=False, sage_options=DD,...
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> Timer().start().stop().annotate(runner)>>> D = DictAsObject({ err :None})>>> runner.update_results(D)0>>> DTR.report(FDS, False, Integer(0), (sum([len(t.examples) for t in␣˓→doctests]), D),...
"Good tests")[... tests, ... s]>>> DTR.stats{ sage.doctest.reporting : { ntests : ..., walltime : ...}}

Or inform the user that some doctests failed:sage: runner.failures = 1
sage: runner.update_results(D)1sage: DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D),....:"Doctest output including the failure...")[... tests, 1 failure, ... s]>>> from sage.all import *>>> runner.failures = Integer(1)>>> runner.update_results(D)1>>> DTR.report(FDS, False, Integer(0), (sum([len(t.examples) for t in␣˓→doctests]), D),...
"Doctest output including the failure...")[... tests, 1 failure, ... s]

If the user has requested that we report on skipped doctests, we do so:sage: DC.options = DocTestDefaults(show_skipped=True)
sage: from collections import defaultdict
sage: optionals = defaultdict(int)sage: optionals[ magma ] = 5; optionals[ long time ] = 4; optionals[ ] = 1;␣
˓→optionals[ not tested ] = 2
sage: D = DictAsObject(dict(err=None, optionals=optionals))sage: runner.failures = 0
sage: runner.update_results(D)0sage: DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D),
˓→ "Good tests")
1 unlabeled test not run
4 long tests not run
5 magma tests not run
2 not tested tests not run
0 tests not run because we ran out of time[... tests, ... s]>>> from sage.all import *>>> DC.options = DocTestDefaults(show_skipped=True)>>> from collections import defaultdict>>> optionals = defaultdict(int)(continues on next page)

106

Chapter 5. Reporting doctest results

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> optionals[ magma ] = Integer(5); optionals[ long time ] = Integer(4);␣
˓→optionals[] = Integer(1); optionals[ not tested ] = Integer(2)>>> D = DictAsObject(dict(err=None, optionals=optionals))>>> runner.failures = Integer(0)>>> runner.update_results(D)0>>> DTR.report(FDS, False, Integer(0), (sum([len(t.examples) for t in␣˓→doctests]), D), "Good tests")
1 unlabeled test not run
4 long tests not run
5 magma tests not run
2 not tested tests not run
0 tests not run because we ran out of time[... tests, ... s]

Test an internal error in the reporter:
sage: DTR.report(None, None, None, None, None)
Traceback (most recent call last):...
AttributeError: NoneType object has no attributebasename ...

>>> from sage.all import *>>> DTR.report(None, None, None, None, None)
Traceback (most recent call last):...
AttributeError: NoneType object has no attributebasename ...

The only-errors mode does not output anything on success:sage: DD = DocTestDefaults(only_errors=True)
sage: FDS = FileDocTestSource(filename, DD)
sage: DC = DocTestController(DD, [filename])
sage: DTR = DocTestReporter(DC)sage: doctests, extras = FDS.create_doctests(globals())
sage: runner = SageDocTestRunner(....:
SageOutputChecker(), verbose=False, sage_options=DD,....:
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: Timer().start().stop().annotate(runner)
sage: D = DictAsObject({ err :None})sage: runner.update_results(D)0sage: DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D),....:"Good tests")>>> from sage.all import *>>> DD = DocTestDefaults(only_errors=True)>>> FDS = FileDocTestSource(filename, DD)>>> DC = DocTestController(DD, [filename])>>> DTR = DocTestReporter(DC)>>> doctests, extras = FDS.create_doctests(globals())>>> runner = SageDocTestRunner(...
SageOutputChecker(), verbose=False, sage_options=DD,...
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)>>> Timer().start().stop().annotate(runner)>>> D = DictAsObject({ err :None})(continues on next page)

107

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> runner.update_results(D)0>>> DTR.report(FDS, False, Integer(0), (sum([len(t.examples) for t in␣˓→doctests]), D),...
"Good tests")

However, failures are still output in the errors-only mode:
sage: runner.failures = 1
sage: runner.update_results(D)1sage: DTR.report(FDS, False, 0, (sum([len(t.examples) for t in doctests]), D),....:"Failed test")[... tests, 1 failure, ... s]>>> from sage.all import *>>> runner.failures = Integer(1)>>> runner.update_results(D)1>>> DTR.report(FDS, False, Integer(0), (sum([len(t.examples) for t in␣˓→doctests]), D),...
"Failed test")[... tests, 1 failure, ... s]report_head(source, fail_msg=None)
Return the sage -t [options] file.py line as string.
INPUT:• source – a source from sage.doctest.sources• fail_msg – None or a string
EXAMPLES:
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.forker import SageDocTestRunner
sage: filename = sage.doctest.reporting.__file__
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename, DD)
sage: DC = DocTestController(DD, [filename])
sage: DTR = DocTestReporter(DC)sage: print(DTR.report_head(FDS))
sage -t .../sage/doctest/reporting.py>>> from sage.all import *>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> from sage.doctest.sources import FileDocTestSource>>> from sage.doctest.forker import SageDocTestRunner>>> filename = sage.doctest.reporting.__file__>>> DD = DocTestDefaults()>>> FDS = FileDocTestSource(filename, DD)>>> DC = DocTestController(DD, [filename])>>> DTR = DocTestReporter(DC)(continues on next page)

108

Chapter 5. Reporting doctest results

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> print(DTR.report_head(FDS))
sage -t .../sage/doctest/reporting.py

The same with various options:sage: DD.long = True
sage: print(DTR.report_head(FDS))
sage -t --long .../sage/doctest/reporting.py
sage: print(DTR.report_head(FDS, "Failed by self-sabotage"))sage -t --long .../sage/doctest/reporting.py # Failed by self-sabotage>>> from sage.all import *>>> DD.long = True>>> print(DTR.report_head(FDS))
sage -t --long .../sage/doctest/reporting.py>>> print(DTR.report_head(FDS, "Failed by self-sabotage"))sage -t --long .../sage/doctest/reporting.py # Failed by self-sabotage

were_doctests_with_optional_tag_run(tag)Return whether doctests marked with this tag were run.
INPUT:• tag – string
EXAMPLES:
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: filename = sage.doctest.reporting.__file__
sage: DC = DocTestController(DocTestDefaults(), [filename])
sage: DTR = DocTestReporter(DC)>>> from sage.all import *>>> from sage.doctest.reporting import DocTestReporter>>> from sage.doctest.control import DocTestController, DocTestDefaults>>> filename = sage.doctest.reporting.__file__>>> DC = DocTestController(DocTestDefaults(), [filename])>>> DTR = DocTestReporter(DC)sage: DTR.were_doctests_with_optional_tag_run( sage )Truesage: DTR.were_doctests_with_optional_tag_run( nice_unavailable_package )False>>> from sage.all import *>>> DTR.were_doctests_with_optional_tag_run( sage )True>>> DTR.were_doctests_with_optional_tag_run( nice_unavailable_package )False

When latex is available, doctests marked with optional tag latex are run by default since Issue #32174:
sage: # needs SAGE_SRC
sage: filename = os.path.join(SAGE_SRC, sage , misc , latex.py )sage: DC = DocTestController(DocTestDefaults(), [filename])(continues on next page)

109

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

sage: DTR = DocTestReporter(DC)sage: DTR.were_doctests_with_optional_tag_run( latex )True# optional - latex>>> from sage.all import *>>> # needs SAGE_SRC>>> filename = os.path.join(SAGE_SRC, sage , misc , latex.py )>>> DC = DocTestController(DocTestDefaults(), [filename])>>> DTR = DocTestReporter(DC)>>> DTR.were_doctests_with_optional_tag_run( latex )# optional - latexTruesage.doctest.reporting.signal_name(sig)Return a string describing a signal number.
EXAMPLES:
sage: from signal import SIGSEGV
sage: from sage.doctest.reporting import signal_namesage: signal_name(SIGSEGV)
segmentation fault
sage: signal_name(9)kill signal
sage: signal_name(12345)
signal 12345>>> from sage.all import *>>> from signal import SIGSEGV>>> from sage.doctest.reporting import signal_name>>> signal_name(SIGSEGV)
segmentation fault>>> signal_name(Integer(9))
kill signal>>> signal_name(Integer(12345))
signal 12345

110

Chapter 5. Reporting doctest results

CHAPTER

SIX

DETECTING EXTERNAL SOFTWARE

This module makes up a list of external software that Sage interfaces. Availability of each software is tested only whennecessary. This is mainly used for the doctests which require certain external software installed on the system.
Even though the functions in this module should also work when an external software is not present, most doctests in thismodule are only tested if testing of external software is explicitly enabled in order to avoid invoking external softwareotherwise. See Issue #28819 for details.
AUTHORS:• Kwankyu Lee (2016-03-09) – initial version, based on code by Robert Bradshaw and Nathann Cohen
class sage.doctest.external.AvailableSoftware
Bases: object
This class keeps the set of available software whose availability is detected lazily from the list of external software.
EXAMPLES:
sage: from sage.doctest.external import external_software, available_software
sage: external_software[ cplex ,dvips ,
ffmpeg ,gurobi ,internet ,latex ,
latex_package_tkz_graph ,lualatex ,macaulay2 ,magma ,
maple ,
mathematica ,
matlab ,octave ,pdflatex ,scilab ,xelatex ]sage: internet in available_software # random, optional - internet
Truesage: available_software.issuperset(set([ internet , latex ])) # random, optional␣
˓→- internet latex
True>>> from sage.all import *>>> from sage.doctest.external import external_software, available_software(continues on next page)

111

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> external_software[ cplex ,dvips ,
ffmpeg ,gurobi ,internet ,latex ,
latex_package_tkz_graph ,lualatex ,macaulay2 ,magma ,
maple ,
mathematica ,
matlab ,octave ,pdflatex ,scilab ,xelatex ]>>> internet in available_software # random, optional - internet
True>>> available_software.issuperset(set([ internet , latex ])) # random, optional -␣˓→internet latexTruedetectable()
Return the list of names of those features for which testing their presence is allowed.
hidden()Return the list of detected hidden external software.
EXAMPLES:
sage: # needs conway_polynomials database_cremona_mini_ellcurve database_
˓→ellcurves database_graphs
sage: from sage.doctest.external import available_software
sage: from sage.features.databases import all_features
sage: for f in all_features():....:f.hide()....:if f._spkg_type() == standard :....:test = f.name in available_software....:
f.unhide()sage: sorted(available_software.hidden())[... conway_polynomials ,...
database_cremona_mini_ellcurve ,...
database_ellcurves ,...
database_graphs ...]>>> from sage.all import *>>> # needs conway_polynomials database_cremona_mini_ellcurve database_
˓→ellcurves database_graphs>>> from sage.doctest.external import available_software>>> from sage.features.databases import all_features>>> for f in all_features():...
f.hide()...
if f._spkg_type() == standard :...
test = f.name in available_software(continues on next page)

112

Chapter 6. Detecting external software

Sage's Doctesting Framework, Release 10.4

(continued from previous page)...
f.unhide()>>> sorted(available_software.hidden())[... conway_polynomials ,...
database_cremona_mini_ellcurve ,...
database_ellcurves ,...
database_graphs ...]

issuperset(other)
Return True if other is a subset of self.
EXAMPLES:
sage: from sage.doctest.external import available_software
sage: available_software.issuperset(set([ internet , latex , magma ])) #␣˓→random, optional - internet latex magma
True>>> from sage.all import *>>> from sage.doctest.external import available_software>>> available_software.issuperset(set([ internet , latex , magma ])) # random,
˓→ optional - internet latex magmaTrueseen()
Return the list of detected external software.
EXAMPLES:
sage: from sage.doctest.external import available_software
sage: available_software.seen() # random[ internet , latex , magma ]>>> from sage.all import *>>> from sage.doctest.external import available_software>>> available_software.seen() # random[ internet , latex , magma ]sage.doctest.external.external_features()
Generate the features that are only to be tested if --optional=external is used.
EXAMPLES:
sage: from sage.doctest.external import external_features
sage: next(external_features())
Feature( internet )>>> from sage.all import *>>> from sage.doctest.external import external_features>>> next(external_features())
Feature( internet )sage.doctest.external.has_4ti2()
Test if the 4ti2 package is available.
EXAMPLES:

113

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.external import has_4ti2
sage: has_4ti2()# optional -- 4ti2
FeatureTestResult( 4ti2 , True)>>> from sage.all import *>>> from sage.doctest.external import has_4ti2>>> has_4ti2()# optional -- 4ti2
FeatureTestResult( 4ti2 , True)sage.doctest.external.has_cplex()
Test if CPLEX is available.
EXAMPLES:
sage: from sage.doctest.external import has_cplex
sage: has_cplex() # random, optional - CPLEX
FeatureTestResult( cplex , True)>>> from sage.all import *>>> from sage.doctest.external import has_cplex>>> has_cplex() # random, optional - CPLEX
FeatureTestResult( cplex , True)sage.doctest.external.has_dvipng()
Test if dvipng is available.
EXAMPLES:
sage: from sage.doctest.external import has_dvipng
sage: has_dvipng() # optional -- dvipng
FeatureTestResult( dvipng , True)>>> from sage.all import *>>> from sage.doctest.external import has_dvipng>>> has_dvipng() # optional -- dvipng
FeatureTestResult( dvipng , True)sage.doctest.external.has_ffmpeg()
Test if ffmpeg is available.
EXAMPLES:
sage: from sage.doctest.external import has_ffmpeg
sage: has_ffmpeg()# optional -- ffmpeg
FeatureTestResult( ffmpeg , True)>>> from sage.all import *>>> from sage.doctest.external import has_ffmpeg>>> has_ffmpeg()# optional -- ffmpeg
FeatureTestResult( ffmpeg , True)sage.doctest.external.has_graphviz()
Test if graphviz (dot, twopi, neato) are available.
EXAMPLES:

114

Chapter 6. Detecting external software

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.external import has_graphviz
sage: has_graphviz()# optional -- graphviz
FeatureTestResult( graphviz , True)>>> from sage.all import *>>> from sage.doctest.external import has_graphviz>>> has_graphviz()# optional -- graphviz
FeatureTestResult( graphviz , True)sage.doctest.external.has_gurobi()
Test if Gurobi is available.
EXAMPLES:
sage: from sage.doctest.external import has_gurobi
sage: has_gurobi() # random, optional - Gurobi
FeatureTestResult( gurobi , True)>>> from sage.all import *>>> from sage.doctest.external import has_gurobi>>> has_gurobi() # random, optional - Gurobi
FeatureTestResult( gurobi , True)sage.doctest.external.has_imagemagick()
Test if ImageMagick (command magick or convert) is available.
EXAMPLES:
sage: from sage.doctest.external import has_imagemagick
sage: has_imagemagick() # optional -- imagemagick
FeatureTestResult( imagemagick , True)>>> from sage.all import *>>> from sage.doctest.external import has_imagemagick>>> has_imagemagick() # optional -- imagemagick
FeatureTestResult( imagemagick , True)sage.doctest.external.has_internet()
Test if Internet is available.
Failure of connecting to the site “https://www.sagemath.org” within a second is regarded as internet being notavailable.
EXAMPLES:
sage: from sage.doctest.external import has_internet
sage: has_internet() # random, optional -- internet
FeatureTestResult( internet , True)>>> from sage.all import *>>> from sage.doctest.external import has_internet>>> has_internet() # random, optional -- internet
FeatureTestResult( internet , True)sage.doctest.external.has_latex()
Test if Latex is available.
115Sage's Doctesting Framework, Release 10.4

EXAMPLES:
sage: from sage.doctest.external import has_latex
sage: has_latex() # optional - latex
FeatureTestResult( latex , True)>>> from sage.all import *>>> from sage.doctest.external import has_latex>>> has_latex() # optional - latex
FeatureTestResult( latex , True)sage.doctest.external.has_lualatex()
Test if lualatex is available.
EXAMPLES:
sage: from sage.doctest.external import has_lualatex
sage: has_lualatex()# optional - lualatex
FeatureTestResult( lualatex , True)>>> from sage.all import *>>> from sage.doctest.external import has_lualatex>>> has_lualatex()# optional - lualatex
FeatureTestResult( lualatex , True)sage.doctest.external.has_macaulay2()
Test if Macaulay2 is available.
EXAMPLES:
sage: from sage.doctest.external import has_macaulay2
sage: has_macaulay2() # random, optional - macaulay2
True>>> from sage.all import *>>> from sage.doctest.external import has_macaulay2>>> has_macaulay2() # random, optional - macaulay2
True

sage.doctest.external.has_magma()
Test if Magma is available.
EXAMPLES:
sage: from sage.doctest.external import has_magma
sage: has_magma() # random, optional - magma
True>>> from sage.all import *>>> from sage.doctest.external import has_magma>>> has_magma() # random, optional - magma
Truesage.doctest.external.has_maple()
Test if Maple is available.
EXAMPLES:
116

Chapter 6. Detecting external software

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.external import has_maple
sage: has_maple() # random, optional - maple
True>>> from sage.all import *>>> from sage.doctest.external import has_maple>>> has_maple() # random, optional - maple
Truesage.doctest.external.has_mathematica()
Test if Mathematica is available.
EXAMPLES:
sage: from sage.doctest.external import has_mathematica
sage: has_mathematica() # random, optional - mathematica
True>>> from sage.all import *>>> from sage.doctest.external import has_mathematica>>> has_mathematica() # random, optional - mathematica
Truesage.doctest.external.has_matlab()
Test if Matlab is available.
EXAMPLES:
sage: from sage.doctest.external import has_matlab
sage: has_matlab() # random, optional - matlab
True>>> from sage.all import *>>> from sage.doctest.external import has_matlab>>> has_matlab() # random, optional - matlab
Truesage.doctest.external.has_octave()
Test if Octave is available.
EXAMPLES:
sage: from sage.doctest.external import has_octave
sage: has_octave() # random, optional - octave
True>>> from sage.all import *>>> from sage.doctest.external import has_octave>>> has_octave() # random, optional - octave
Truesage.doctest.external.has_pandoc()
Test if pandoc is available.
EXAMPLES:

117

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.external import has_pandoc
sage: has_pandoc()# optional -- pandoc
FeatureTestResult( pandoc , True)>>> from sage.all import *>>> from sage.doctest.external import has_pandoc>>> has_pandoc()# optional -- pandoc
FeatureTestResult( pandoc , True)sage.doctest.external.has_pdf2svg()
Test if pdf2svg is available.
EXAMPLES:
sage: from sage.doctest.external import has_pdf2svg
sage: has_pdf2svg() # optional -- pdf2svg
FeatureTestResult( pdf2svg , True)>>> from sage.all import *>>> from sage.doctest.external import has_pdf2svg>>> has_pdf2svg() # optional -- pdf2svg
FeatureTestResult( pdf2svg , True)sage.doctest.external.has_pdflatex()
Test if pdflatex is available.
EXAMPLES:
sage: from sage.doctest.external import has_pdflatex
sage: has_pdflatex()# optional - pdflatex
FeatureTestResult( pdflatex , True)>>> from sage.all import *>>> from sage.doctest.external import has_pdflatex>>> has_pdflatex()# optional - pdflatex
FeatureTestResult( pdflatex , True)sage.doctest.external.has_rubiks()
Test if the rubiks package (cu2, cubex, dikcube, mcube, optimal, and size222) is available.
EXAMPLES:
sage: from sage.doctest.external import has_rubiks
sage: has_rubiks()# optional -- rubiks
FeatureTestResult( rubiks , True)>>> from sage.all import *>>> from sage.doctest.external import has_rubiks>>> has_rubiks()# optional -- rubiks
FeatureTestResult( rubiks , True)sage.doctest.external.has_scilab()
Test if Scilab is available.
EXAMPLES:

118

Chapter 6. Detecting external software

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.external import has_scilab
sage: has_scilab() # random, optional - scilab
True>>> from sage.all import *>>> from sage.doctest.external import has_scilab>>> has_scilab() # random, optional - scilabTruesage.doctest.external.has_xelatex()
Test if xelatex is available.
EXAMPLES:
sage: from sage.doctest.external import has_xelatex
sage: has_xelatex()# optional - xelatex
FeatureTestResult( xelatex , True)>>> from sage.all import *>>> from sage.doctest.external import has_xelatex>>> has_xelatex()# optional - xelatex
FeatureTestResult( xelatex , True)

119

Sage's Doctesting Framework, Release 10.4

120

Chapter 6. Detecting external software

CHAPTERSEVENTEST THE DOCTESTING FRAMEWORK

Many tests (with expected failures or crashes) are run in a subprocess, those tests can be found in the tests/ subdirectory.
EXAMPLES:
sage: import signal
sage: import subprocess
sage: import time
sage: from sage.env import SAGE_SRC
sage: tests_dir = os.path.join(SAGE_SRC,
sage: tests_env = dict(os.environ)>>> from sage.all import *>>> import signal>>> import subprocess>>> import time>>> from sage.env import SAGE_SRC>>> tests_dir = os.path.join(SAGE_SRC,>>> tests_env = dict(os.environ)sage ,

sage ,

doctest ,doctest ,tests )

tests )

Unset TERM when running doctests, see Issue #14370:
sage: try:....:
del tests_env[ TERM ]....: except KeyError:
....:
pass
sage: kwds = { cwd : tests_dir,>>> from sage.all import *>>> try:...
del tests_env[ TERM ]... except KeyError:...
pass>>> kwds = { cwd : tests_dir,

env :tests_env}

env :tests_env}

Check that Issue #2235 has been fixed:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "longtime.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 longtime.rst[0 tests, ...s](continues on next page)

121

Sage's Doctesting Framework, Release 10.4

(continued from previous page)---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "-l", "longtime.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --long --warn-long 0.0 --random-seed=0 longtime.rst[1 test, ...s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "longtime.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 longtime.rst[0 tests, ...s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "-l", "longtime.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --long --warn-long 0.0 --random-seed=0 longtime.rst[1 test, ...s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0

Check handling of tolerances:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "tolerance.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 tolerance.rst**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print(":-(")# abs tol 0.1
Expected:
:-)
Got:
:-(
(continues on next page)

122

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("1.0 2.0 3.0") # abs tol 0.1
Expected:
4.0 5.0Got:
1.0 2.0 3.0**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("Hello") # abs tol 0.1
Expected:
1.0
Got:
Hello
**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("1.0") # abs tol 0.1
Expected:
Hello
Got:
1.0
**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("Hello 1.1") # abs tol 0.1
Expected:
Goodbye 1.0
Got:
Hello 1.1**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("Hello 1.0") # rel tol 1e-6
Expected:
Goodbye 0.999999
Got:
Hello 1.0
Tolerance exceeded:
0.999999 vs 1.0, tolerance 2e-6 > 1e-6**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("Hello 1.0") # rel tol 1e-6
Expected:
Hello ...
Got:
Hello 1.0
Note: combining tolerance (# tol) with ellipsis (...) is not supported**********************************************************************...
1>>> from sage.all import *(continues on next page)

123

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "tolerance.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 tolerance.rst**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print(":-(")# abs tol 0.1
Expected:
:-)
Got:
:-(
**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("1.0 2.0 3.0") # abs tol 0.1
Expected:
4.0 5.0Got:
1.0 2.0 3.0**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("Hello") # abs tol 0.1
Expected:
1.0
Got:
Hello
**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("1.0") # abs tol 0.1
Expected:
Hello
Got:
1.0
**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("Hello 1.1") # abs tol 0.1
Expected:
Goodbye 1.0
Got:
Hello 1.1**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance
Failed example:
print("Hello 1.0") # rel tol 1e-6
Expected:
Goodbye 0.999999
Got:
Hello 1.0
Tolerance exceeded:
0.999999 vs 1.0, tolerance 2e-6 > 1e-6**********************************************************************
File "tolerance.rst", line ..., in sage.doctest.tests.tolerance(continues on next page)

124

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

Failed example:
print("Hello 1.0") # rel tol 1e-6
Expected:
Hello ...
Got:
Hello 1.0
Note: combining tolerance (# tol) with ellipsis (...) is not supported**********************************************************************...
1

Test the --initial option:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "-i", "initial.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 initial.rst**********************************************************************
File "initial.rst", line 4, in sage.doctest.tests.initial
Failed example:
a = binomiak(10,5) # random to test that we still get the exception
Exception raised:
Traceback (most recent call last):...
NameError: name binomiak is not defined**********************************************************************
File "initial.rst", line 14, in sage.doctest.tests.initial
Failed example:
binomial(10,5)
Expected:
255
Got:
252
**********************************************************************...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 initial.rst # 5 doctests failed---------------------------------------------------------------------...
1>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "-i", "initial.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 initial.rst**********************************************************************
File "initial.rst", line 4, in sage.doctest.tests.initial
Failed example:
a = binomiak(10,5) # random to test that we still get the exception
Exception raised:
Traceback (most recent call last):...
NameError: name binomiak is not defined(continues on next page)

125

Sage's Doctesting Framework, Release 10.4

(continued from previous page)**********************************************************************
File "initial.rst", line 14, in sage.doctest.tests.initial
Failed example:
binomial(10,5)
Expected:
255
Got:
252
**********************************************************************...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 initial.rst # 5 doctests failed---------------------------------------------------------------------...
1

Test the --exitfirst option:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "--exitfirst", "initial.rst"],␣
˓→**kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 initial.rst**********************************************************************
File "initial.rst", line 4, in sage.doctest.tests.initial
Failed example:
a = binomiak(10,5) # random to test that we still get the exception
Exception raised:
Traceback (most recent call last):...
NameError: name binomiak is not defined**********************************************************************...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 initial.rst # 1 doctest failed---------------------------------------------------------------------...
1>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "--exitfirst", "initial.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 initial.rst**********************************************************************
File "initial.rst", line 4, in sage.doctest.tests.initial
Failed example:
a = binomiak(10,5) # random to test that we still get the exception
Exception raised:
Traceback (most recent call last):...
NameError: name binomiak is not defined**********************************************************************...
(continues on next page)

126

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 initial.rst # 1 doctest failed---------------------------------------------------------------------...
1

Test a timeout using the SAGE_TIMEOUT environment variable. Also set CYSIGNALS_CRASH_NDEBUG to help
ensure the test times out in a timely manner (Issue #26912):
sage: from copy import deepcopy
sage: kwds2 = deepcopy(kwds)
sage: kwds2[ env ].update({ SAGE_TIMEOUT : 1 , CYSIGNALS_CRASH_NDEBUG : 1 })sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "99seconds.rst"], **kwds2)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 99seconds.rst
Timed out**********************************************************************
Tests run before process (pid=...) timed out:...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 99seconds.rst # Timed out---------------------------------------------------------------------...
4>>> from sage.all import *>>> from copy import deepcopy>>> kwds2 = deepcopy(kwds)>>> kwds2[ env ].update({ SAGE_TIMEOUT : 1 , CYSIGNALS_CRASH_NDEBUG : 1 })>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "99seconds.rst"], **kwds2)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 99seconds.rst
Timed out**********************************************************************
Tests run before process (pid=...) timed out:...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 99seconds.rst # Timed out---------------------------------------------------------------------...
4

Test handling of KeyboardInterrupt in doctests:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "keyboardinterrupt.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 keyboardinterrupt.rst**********************************************************************
File "keyboardinterrupt.rst", line 11, in sage.doctest.tests.keyboardinterrupt
Failed example:(continues on next page)

127

Sage's Doctesting Framework, Release 10.4

(continued from previous page)raise KeyboardInterrupt
Exception raised:
Traceback (most recent call last):...
KeyboardInterrupt**********************************************************************...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 keyboardinterrupt.rst # 1 doctest failed---------------------------------------------------------------------...
1>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "keyboardinterrupt.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 keyboardinterrupt.rst**********************************************************************
File "keyboardinterrupt.rst", line 11, in sage.doctest.tests.keyboardinterrupt
Failed example:
raise KeyboardInterrupt
Exception raised:
Traceback (most recent call last):...
KeyboardInterrupt**********************************************************************...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 keyboardinterrupt.rst # 1 doctest failed---------------------------------------------------------------------...
1

Interrupt the doctester:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "interrupt.rst"], **kwds)
Running doctests...
Doctesting 1 file.
Killing test interrupt.rst---------------------------------------------------------------------Doctests interrupted: 0/1 files tested---------------------------------------------------------------------...
128>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "interrupt.rst"], **kwds)
Running doctests...
Doctesting 1 file.
Killing test interrupt.rst---------------------------------------------------------------------Doctests interrupted: 0/1 files tested(continues on next page)

128

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)---------------------------------------------------------------------...
128

Interrupt the doctester (while parallel testing) when a doctest cannot be interrupted. We also test that passing a ridiculous
number of threads doesn’t hurt:
sage: F = tmp_filename()
sage: from copy import deepcopy
sage: kwds2 = deepcopy(kwds)
sage: kwds2[ env ][ DOCTEST_TEST_PID_FILE ] = F # Doctester will write its PID in␣˓→this file
sage: subprocess.call(["sage", "-tp", "1000000", "--timeout=120", # long time....:"--die_timeout=10", "--optional=sage",....:"--warn-long", "0", "99seconds.rst", "interrupt_diehard.rst"], **kwds2)
Running doctests...
Doctesting 2 files using 1000000 threads...
Killing test 99seconds.rst
Killing test interrupt_diehard.rst---------------------------------------------------------------------Doctests interrupted: 0/2 files tested---------------------------------------------------------------------...
128>>> from sage.all import *>>> F = tmp_filename()>>> from copy import deepcopy>>> kwds2 = deepcopy(kwds)>>> kwds2[ env ][ DOCTEST_TEST_PID_FILE ] = F # Doctester will write its PID in this␣˓→file>>> subprocess.call(["sage", "-tp", "1000000", "--timeout=120", # long time...
"--die_timeout=10", "--optional=sage",...
"--warn-long", "0", "99seconds.rst", "interrupt_diehard.rst"], **kwds2)
Running doctests...
Doctesting 2 files using 1000000 threads...
Killing test 99seconds.rst
Killing test interrupt_diehard.rst---------------------------------------------------------------------Doctests interrupted: 0/2 files tested---------------------------------------------------------------------...
128

Even though the doctester master process has exited, the child process is still alive, but it should be killed automaticallyafter the die_timeout given above (10 seconds):
sage: # long time
sage: pid = int(open(F).read())
sage: time.sleep(2)sage: os.kill(pid, signal.SIGQUIT)
sage: time.sleep(8)sage: os.kill(pid, signal.SIGQUIT)
Traceback (most recent call last):...
ProcessLookupError: ...

# 2 seconds passed => still alive# 10 seconds passed => dead# random

129

Sage's Doctesting Framework, Release 10.4>>> from sage.all import *>>> # long time>>> pid = int(open(F).read())>>> time.sleep(Integer(2))>>> os.kill(pid, signal.SIGQUIT) # 2 seconds passed => still alive>>> time.sleep(Integer(8))>>> os.kill(pid, signal.SIGQUIT) # 10 seconds passed => dead # random
Traceback (most recent call last):...
ProcessLookupError: ...

If the child process is dead and removed, the last output should be as above. However, the child process interrupted its
parent process (see "interrupt_diehard.rst"), and became an orphan process. Depending on the system, anorphan process may eventually become a zombie process instead of being removed, and then the last output would justbe a blank. Hence the # random tag.
Test a doctest failing with abort():sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "abort.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 abort.rst
Killed due to abort**********************************************************************
Tests run before process (pid=...) failed:...
-----------------------------------------------------------------------Unhandled SIGABRT: An abort() occurred.
This probably occurred because a *compiled* module has a bugin it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
-----------------------------------------------------------------------...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 abort.rst # Killed due to abort---------------------------------------------------------------------...
16>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "abort.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 abort.rst
Killed due to abort**********************************************************************
Tests run before process (pid=...) failed:...
-----------------------------------------------------------------------Unhandled SIGABRT: An abort() occurred.
This probably occurred because a *compiled* module has a bugin it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.
-----------------------------------------------------------------------...
(continues on next page)

130

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 abort.rst # Killed due to abort---------------------------------------------------------------------...
16

A different kind of crash:sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "fail_and_die.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 fail_and_die.rst**********************************************************************
File "fail_and_die.rst", line 5, in sage.doctest.tests.fail_and_die
Failed example:
this_gives_a_NameError
Exception raised:
Traceback (most recent call last):...
NameError: name this_gives_a_NameError is not defined
Killed due to kill signal**********************************************************************
Tests run before process (pid=...) failed:...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 fail_and_die.rst # Killed due to kill signal---------------------------------------------------------------------...
16>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "fail_and_die.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 fail_and_die.rst**********************************************************************
File "fail_and_die.rst", line 5, in sage.doctest.tests.fail_and_die
Failed example:
this_gives_a_NameError
Exception raised:
Traceback (most recent call last):...
NameError: name this_gives_a_NameError is not defined
Killed due to kill signal**********************************************************************
Tests run before process (pid=...) failed:...
---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 fail_and_die.rst # Killed due to kill signal---------------------------------------------------------------------...
16

Test that sig_on_count is checked correctly:131Sage's Doctesting Framework, Release 10.4

sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "sig_on.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 sig_on.rst**********************************************************************
File "sig_on.rst", line 6, in sage.doctest.tests.sig_on
Failed example:
sig_on_count() # check sig_on/off pairings (virtual doctest)
Expected:0
Got:1**********************************************************************
1 item had failures:
1 of
5 in sage.doctest.tests.sig_on[3 tests, 1 failure, ...]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 sig_on.rst # 1 doctest failed---------------------------------------------------------------------...
1>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "sig_on.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 sig_on.rst**********************************************************************
File "sig_on.rst", line 6, in sage.doctest.tests.sig_on
Failed example:
sig_on_count() # check sig_on/off pairings (virtual doctest)
Expected:0
Got:1**********************************************************************
1 item had failures:
1 of
5 in sage.doctest.tests.sig_on[3 tests, 1 failure, ...]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 sig_on.rst # 1 doctest failed---------------------------------------------------------------------...
1

Test logfiles in serial and parallel mode (see Issue #19271):
sage: t = tmp_filename()
sage: subprocess.call(["sage", "-t", "--serial", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "--logfile", t, "simple_failure.rst
˓→"],
....:
stdout=open(os.devnull, "w"), **kwds)
1
sage: print(open(t).read()) # long time
Running doctests...
(continues on next page)

132

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst**********************************************************************
File "simple_failure.rst", line 7, in sage.doctest.tests.simple_failure
Failed example:a * b
Expected:
20
Got:
15
**********************************************************************
1 item had failures:
1 of
5 in sage.doctest.tests.simple_failure[4 tests, 1 failure, ...]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst # 1 doctest failed---------------------------------------------------------------------...
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "--logfile", t, "simple_failure.rst
˓→"],
....:
stdout=open(os.devnull, "w"), **kwds)
1
sage: print(open(t).read()) # long time
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst**********************************************************************
File "simple_failure.rst", line 7, in sage.doctest.tests.simple_failure
Failed example:a * b
Expected:
20
Got:
15
**********************************************************************
1 item had failures:
1 of
5 in sage.doctest.tests.simple_failure[4 tests, 1 failure, ...]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst # 1 doctest failed---------------------------------------------------------------------...
>>> from sage.all import *>>> t = tmp_filename()>>> subprocess.call(["sage", "-t", "--serial", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "--logfile", t, "simple_failure.rst"],...
stdout=open(os.devnull, "w"), **kwds)
1>>> print(open(t).read()) # long time
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst**********************************************************************(continues on next page)

133

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

File "simple_failure.rst", line 7, in sage.doctest.tests.simple_failure
Failed example:a * b
Expected:
20
Got:
15
**********************************************************************
1 item had failures:
1 of
5 in sage.doctest.tests.simple_failure[4 tests, 1 failure, ...]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst # 1 doctest failed---------------------------------------------------------------------...
>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "--logfile", t, "simple_failure.rst"],...
stdout=open(os.devnull, "w"), **kwds)
1>>> print(open(t).read()) # long time
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst**********************************************************************
File "simple_failure.rst", line 7, in sage.doctest.tests.simple_failure
Failed example:a * b
Expected:
20
Got:
15
**********************************************************************
1 item had failures:
1 of
5 in sage.doctest.tests.simple_failure[4 tests, 1 failure, ...]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst # 1 doctest failed---------------------------------------------------------------------...

Test the --debug option:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "--debug", "simple_failure.rst"],....:stdin=open(os.devnull), **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst**********************************************************************
File "simple_failure.rst", line 7, in sage.doctest.tests.simple_failure
Failed example:a * b
Expected:
20
Got:
15
(continues on next page)

134

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)**********************************************************************
Previously executed commands:
s...: a = 3s...: b = 5s...: a + b
8sage:
Returning to doctests...
**********************************************************************
1 item had failures:
1 of
5 in sage.doctest.tests.simple_failure[4 tests, 1 failure, ...]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst # 1 doctest failed---------------------------------------------------------------------...
1>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "--debug", "simple_failure.rst"],...
stdin=open(os.devnull), **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst**********************************************************************
File "simple_failure.rst", line 7, in sage.doctest.tests.simple_failure
Failed example:a * b
Expected:
20
Got:
15
**********************************************************************
Previously executed commands:
s...: a = 3s...: b = 5s...: a + b
8
sage:<BLANKLINE>
Returning to doctests...
**********************************************************************
1 item had failures:
1 of
5 in sage.doctest.tests.simple_failure[4 tests, 1 failure, ...]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 simple_failure.rst # 1 doctest failed---------------------------------------------------------------------...
1

Test running under gdb, without and with a timeout:
sage: subprocess.call(["sage", "-t","--warn-long", "0",# long time, optional: gdb(continues on next page)

135

Sage's Doctesting Framework, Release 10.4

(continued from previous page)....:"--random-seed=0", "--optional=sage", "--gdb", "1second.rst"],....:stdin=open(os.devnull), **kwds)
exec gdb ...
Running doctests...
Doctesting 1 file...
sage -t... 1second.rst...
[2 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time, optional: gdb...
"--random-seed=0", "--optional=sage", "--gdb", "1second.rst"],...
stdin=open(os.devnull), **kwds)
exec gdb ...
Running doctests...
Doctesting 1 file...
sage -t... 1second.rst...
[2 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0

gdb might need a long time to start up, so we allow 30 seconds:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time, optional: gdb....:"--random-seed=0", "--optional=sage", "--gdb", "-T30", "99seconds.rst"],....:stdin=open(os.devnull), **kwds)
exec gdb ...
Running doctests...
Timed out
4>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time, optional: gdb...
"--random-seed=0", "--optional=sage", "--gdb", "-T30", "99seconds.rst"],...
stdin=open(os.devnull), **kwds)
exec gdb ...
Running doctests...
Timed out
4

Test the --show-skipped option:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "--show-skipped", "show_skipped.rst
˓→"], **kwds)
Running doctests ...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 show_skipped.rst
2 tests not run due to known bugs
(continues on next page)

136

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

1 gap test not run
1 long test not run
1 not tested test not run
0 tests not run because we ran out of time
[2 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "--show-skipped", "show_skipped.rst"],␣
˓→**kwds)
Running doctests ...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 show_skipped.rst
2 tests not run due to known bugs
1 gap test not run
1 long test not run
1 not tested test not run
0 tests not run because we ran out of time
[2 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0

Optional tests are run correctly:
sage: subprocess.call(["sage", "-t", "--warn-long", "0", "--long", # long time....:"--random-seed=0", "--show-skipped", "--optional=sage,gap", "show_skipped.
˓→rst"], **kwds)
Running doctests ...
Doctesting 1 file.
sage -t --long --warn-long 0.0 --random-seed=0 show_skipped.rst
2 tests not run due to known bugs
1 not tested test not run
0 tests not run because we ran out of time
[4 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0sage: subprocess.call(["sage", "-t", "--warn-long", "0", "--long", # long time....:"--random-seed=0", "--show-skipped", "--optional=gAp", "show_skipped.rst"],˓→ **kwds)
Running doctests ...
Doctesting 1 file.
sage -t --long --warn-long 0.0 --random-seed=0 show_skipped.rst
2 tests not run due to known bugs
1 not tested test not run(continues on next page)

137

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

2 sage tests not run
0 tests not run because we ran out of time
[2 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0", "--long", # long time...
"--random-seed=0", "--show-skipped", "--optional=sage,gap", "show_skipped.rst
˓→"], **kwds)
Running doctests ...
Doctesting 1 file.
sage -t --long --warn-long 0.0 --random-seed=0 show_skipped.rst
2 tests not run due to known bugs
1 not tested test not run
0 tests not run because we ran out of time
[4 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0>>> subprocess.call(["sage", "-t", "--warn-long", "0", "--long", # long time...
"--random-seed=0", "--show-skipped", "--optional=gAp", "show_skipped.rst"],␣
˓→**kwds)
Running doctests ...
Doctesting 1 file.
sage -t --long --warn-long 0.0 --random-seed=0 show_skipped.rst
2 tests not run due to known bugs
1 not tested test not run
2 sage tests not run
0 tests not run because we ran out of time
[2 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0

Test an invalid value for --optional:
sage: subprocess.call(["sage", "-t", "--warn-long", "0",....:"--random-seed=0", "--optional=bad-option", "show_skipped.rst"], **kwds)
Traceback (most recent call last):...
ValueError: invalid optional tag bad-option
1>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",...
"--random-seed=0", "--optional=bad-option", "show_skipped.rst"], **kwds)
Traceback (most recent call last):(continues on next page)

138

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)...
ValueError: invalid optional tag
1

bad-option

Test atexit support in the doctesting framework:
sage: F = tmp_filename()
sage: os.path.isfile(F)True
sage: from copy import deepcopy
sage: kwds2 = deepcopy(kwds)
sage: kwds2[ env ][ DOCTEST_DELETE_FILE ] = F
sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "atexit.rst"], **kwds2)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 atexit.rst[3 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0sage: os.path.isfile(F) # long time
False
sage: try:....:
os.unlink(F)....: except OSError:
....:
pass
>>> from sage.all import *>>> F = tmp_filename()>>> os.path.isfile(F)True>>> from copy import deepcopy>>> kwds2 = deepcopy(kwds)>>> kwds2[ env ][ DOCTEST_DELETE_FILE ] = F>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "atexit.rst"], **kwds2)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 atexit.rst[3 tests, ... s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0>>> os.path.isfile(F) # long time
False>>> try:...
os.unlink(F)... except OSError:
...
pass

Test that random tests are reproducible:

139

Sage's Doctesting Framework, Release 10.4

sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=0", "--optional=sage", "random_seed.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 random_seed.rst**********************************************************************
File "random_seed.rst", line 3, in sage.doctest.tests.random_seed
Failed example:
randint(5, 10)
Expected:9
Got:5**********************************************************************
1 item had failures:
1 of
2 in sage.doctest.tests.random_seed[1 test, 1 failure, ...s]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 random_seed.rst # 1 doctest failed---------------------------------------------------------------------...
1sage: subprocess.call(["sage", "-t", "--warn-long", "0",# long time....:"--random-seed=1", "--optional=sage", "random_seed.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=1 random_seed.rst[1 test, ...s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0>>> from sage.all import *>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=0", "--optional=sage", "random_seed.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=0 random_seed.rst**********************************************************************
File "random_seed.rst", line 3, in sage.doctest.tests.random_seed
Failed example:
randint(5, 10)
Expected:9
Got:5**********************************************************************
1 item had failures:
1 of
2 in sage.doctest.tests.random_seed[1 test, 1 failure, ...s]---------------------------------------------------------------------sage -t --warn-long 0.0 --random-seed=0 random_seed.rst # 1 doctest failed---------------------------------------------------------------------...
1(continues on next page)

140

Chapter 7. Test the doctesting framework

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> subprocess.call(["sage", "-t", "--warn-long", "0",# long time...
"--random-seed=1", "--optional=sage", "random_seed.rst"], **kwds)
Running doctests...
Doctesting 1 file.
sage -t --warn-long 0.0 --random-seed=1 random_seed.rst[1 test, ...s]---------------------------------------------------------------------All tests passed!
---------------------------------------------------------------------...
0141

Sage's Doctesting Framework, Release 10.4

142

Chapter 7. Test the doctesting framework

CHAPTEREIGHTUTILITY FUNCTIONSThis module contains various utility functions and classes used in doctesting.
AUTHORS:• David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
class sage.doctest.util.NestedName(base)
Bases: object
Class used to construct fully qualified names based on indentation level.
EXAMPLES:
sage: from sage.doctest.util import NestedName
sage: qname = NestedName( sage.categories.algebras )sage: qname[0] = Algebras ; qname
sage.categories.algebras.Algebras
sage: qname[4] = __contains__ ; qname
sage.categories.algebras.Algebras.__contains__
sage: qname[4] = ParentMethods
sage: qname[8] = from_base_ring ; qname
sage.categories.algebras.Algebras.ParentMethods.from_base_ring>>> from sage.all import *>>> from sage.doctest.util import NestedName>>> qname = NestedName( sage.categories.algebras )>>> qname[Integer(0)] = Algebras ; qname
sage.categories.algebras.Algebras>>> qname[Integer(4)] = __contains__ ; qname
sage.categories.algebras.Algebras.__contains__>>> qname[Integer(4)] = ParentMethods>>> qname[Integer(8)] = from_base_ring ; qname
sage.categories.algebras.Algebras.ParentMethods.from_base_ringclass sage.doctest.util.RecordingDict(*args, **kwds)
Bases: dictThis dictionary is used for tracking the dependencies of an example.
This feature allows examples in different doctests to be grouped for better timing data. It’s obtained by recording
whenever anything is set or retrieved from this dictionary.
EXAMPLES:

143

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.util import RecordingDictsage: D = RecordingDict(test=17)
sage: D.got
set()
sage: D[ test ]17
sage: D.got{ test }sage: D.set
set()
sage: D[ a ] = 1
sage: D[ a ]1sage: D.set{ a }sage: D.got{ test }>>> from sage.all import *>>> from sage.doctest.util import RecordingDict>>> D = RecordingDict(test=Integer(17))>>> D.got
set()>>> D[ test ]17>>> D.got{ test }>>> D.set
set()>>> D[ a ] = Integer(1)>>> D[ a ]1>>> D.set{ a }>>> D.got{ test }copy()
Note that set and got are not copied.
EXAMPLES:
sage: from sage.doctest.util import RecordingDictsage: D = RecordingDict(d = 42)
sage: D[ a ] = 4
sage: D.set{ a }sage: E = D.copy()
sage: E.set
set()
sage: sorted(E.keys())[ a , d ]>>> from sage.all import *>>> from sage.doctest.util import RecordingDict>>> D = RecordingDict(d = Integer(42))>>> D[ a ] = Integer(4)(continues on next page)

144

Chapter 8. Utility functions

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> D.set{ a }>>> E = D.copy()>>> E.set
set()>>> sorted(E.keys())[ a , d ]get(name, default=None)
EXAMPLES:
sage: from sage.doctest.util import RecordingDictsage: D = RecordingDict(d = 42)
sage: D.get( d )42sage: D.got{ d }sage: D.get( not_here )sage: sorted(list(D.got))[ d , not_here ]>>> from sage.all import *>>> from sage.doctest.util import RecordingDict>>> D = RecordingDict(d = Integer(42))>>> D.get( d )42>>> D.got{ d }>>> D.get( not_here )>>> sorted(list(D.got))[ d , not_here ]start()
We track which variables have been set or retrieved. This function initializes these lists to be empty.
EXAMPLES:
sage: from sage.doctest.util import RecordingDictsage: D = RecordingDict(d = 42)
sage: D.set
set()
sage: D[ a ] = 4
sage: D.set{ a }sage: D.start(); D.set
set()>>> from sage.all import *>>> from sage.doctest.util import RecordingDict>>> D = RecordingDict(d = Integer(42))>>> D.set
set()>>> D[ a ] = Integer(4)>>> D.set{ a }(continues on next page)

145

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> D.start(); D.set
set()

class sage.doctest.util.Timer
Bases: object
A simple timer.
EXAMPLES:
sage: from sage.doctest.util import Timersage: Timer(){}sage: TestSuite(Timer()).run()>>> from sage.all import *>>> from sage.doctest.util import Timer>>> Timer(){}>>> TestSuite(Timer()).run()annotate(object)
Annotates the given object with the cputime and walltime stored in this timer.
EXAMPLES:
sage: from sage.doctest.util import Timersage: Timer().start().annotate(EllipticCurve)
sage: EllipticCurve.cputime # random
2.817255
sage: EllipticCurve.walltime # random
1332649288.410404>>> from sage.all import *>>> from sage.doctest.util import Timer>>> Timer().start().annotate(EllipticCurve)>>> EllipticCurve.cputime # random
2.817255>>> EllipticCurve.walltime # random
1332649288.410404

start()Start the timer.
Can be called multiple times to reset the timer.
EXAMPLES:
sage: from sage.doctest.util import Timersage: Timer().start(){ cputime : ..., walltime : ...}>>> from sage.all import *>>> from sage.doctest.util import Timer>>> Timer().start(){ cputime : ..., walltime : ...}

146

Chapter 8. Utility functions

Sage's Doctesting Framework, Release 10.4

stop()
Stops the timer, recording the time that has passed since it was started.
EXAMPLES:
sage: from sage.doctest.util import Timersage: import time
sage: timer = Timer().start()sage: time.sleep(float(0.5))
sage: timer.stop(){ cputime : ..., walltime : ...}>>> from sage.all import *>>> from sage.doctest.util import Timer>>> import time>>> timer = Timer().start()>>> time.sleep(float(RealNumber( 0.5 )))>>> timer.stop(){ cputime : ..., walltime : ...}

sage.doctest.util.count_noun(number, noun, plural=None, pad_number=False, pad_noun=False)
EXAMPLES:
sage: from sage.doctest.util import count_noun
sage: count_noun(1, "apple")
1 applesage: count_noun(1, "apple", pad_noun=True)
1 applesage: count_noun(1, "apple", pad_number=3)
1 applesage: count_noun(2, "orange")
2 oranges
sage: count_noun(3, "peach", "peaches")
3 peachessage: count_noun(1, "peach", plural="peaches", pad_noun=True)
1 peach>>> from sage.all import *>>> from sage.doctest.util import count_noun>>> count_noun(Integer(1), "apple")
1 apple>>> count_noun(Integer(1), "apple", pad_noun=True)
1 apple>>> count_noun(Integer(1), "apple", pad_number=Integer(3))
1 apple>>> count_noun(Integer(2), "orange")
2 oranges>>> count_noun(Integer(3), "peach", "peaches")
3 peaches>>> count_noun(Integer(1), "peach", plural="peaches", pad_noun=True)
1 peachsage.doctest.util.dict_difference(self, other)
Return a dict with all key-value pairs occurring in self but not in other.
EXAMPLES:

147

Sage's Doctesting Framework, Release 10.4

sage: from sage.doctest.util import dict_difference
sage: d1 = {1: a , 2: b , 3: c }sage: d2 = {1: a , 2: x , 4: c }sage: dict_difference(d2, d1){2: x , 4: c }>>> from sage.all import *>>> from sage.doctest.util import dict_difference>>> d1 = {Integer(1): a , Integer(2): b , Integer(3):>>> d2 = {Integer(1): a , Integer(2): x , Integer(4):>>> dict_difference(d2, d1){2: x , 4: c }

c }
c }

sage: from sage.doctest.control import DocTestDefaults
sage: D1 = DocTestDefaults()
sage: D2 = DocTestDefaults(foobar="hello", timeout=100)
sage: dict_difference(D2.__dict__, D1.__dict__){ foobar : hello , timeout : 100}>>> from sage.all import *>>> from sage.doctest.control import DocTestDefaults>>> D1 = DocTestDefaults()>>> D2 = DocTestDefaults(foobar="hello", timeout=Integer(100))>>> dict_difference(D2.__dict__, D1.__dict__){ foobar : hello , timeout : 100}

sage.doctest.util.make_recording_dict(D, st, gt)
Auxiliary function for pickling.
EXAMPLES:
sage: from sage.doctest.util import make_recording_dict
sage: D = make_recording_dict({ a :4, d :42},set([]),set([ not_here ]))sage: sorted(D.items())[( a , 4), ( d , 42)]
sage: D.got{ not_here }>>> from sage.all import *>>> from sage.doctest.util import make_recording_dict>>> D = make_recording_dict({ a :Integer(4), d :Integer(42)},set([]),set([ not_˓→here ]))>>> sorted(D.items())[( a , 4), ( d , 42)]>>> D.got{ not_here }148

Chapter 8. Utility functions

CHAPTERNINEFIXTURES TO HELP TESTING FUNCTIONALITY

Utilities which modify or replace code to help with doctesting functionality. Wrappers, proxies and mockups are typicalexamples of fixtures.
AUTHORS:• Martin von Gagern (2014-12-15): AttributeAccessTracerProxy and trace_method• Martin von Gagern (2015-01-02): Factor out TracerHelper and reproducible_repr
EXAMPLES:
You can use trace_method() to see how a method communicates with its surroundings:
sage: class Foo():....:def f(self):....:
self.y = self.g(self.x)....:
def g(self, arg):....:
return arg + 1....:
sage: foo = Foo()
sage: foo.x = 3
sage: from sage.doctest.fixtures import trace_method
sage: trace_method(foo, "f")sage: foo.f()enter f()read x = 3
call g(3) -> 4
write y = 4
exit f -> None>>> from sage.all import *>>> class Foo():...
def f(self):...
self.y = self.g(self.x)...
def g(self, arg):...
return arg + Integer(1)....:>>> foo = Foo()>>> foo.x = Integer(3)>>> from sage.doctest.fixtures import trace_method>>> trace_method(foo, "f")>>> foo.f()enter f()read x = 3
call g(3) -> 4
(continues on next page)

149

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

write y = 4
exit f -> Noneclass sage.doctest.fixtures.AttributeAccessTracerHelper(delegate, prefix=' ', reads=True)
Bases: object
Helper to print proxied access to attributes.
This class does the actual printing of access traces for objects proxied by AttributeAccessTracerProxy.
The fact that it’s not a proxy at the same time helps avoiding complicated attribute access syntax.
INPUT:• delegate – the actual object to be proxied.
• prefix – (default: " ") string to prepend to each printed output.
• reads – (default: True) whether to trace read access as well.
EXAMPLES:
sage: class Foo():....:def f(self, *args):....:
return self.x*self.x....:
sage: foo = Foo()
sage: from sage.doctest.fixtures import AttributeAccessTracerHelper
sage: pat = AttributeAccessTracerHelper(foo)sage: pat.set("x", 2)
write x = 2
sage: pat.get("x")
read x = 2
2
sage: pat.get("f")(3)call f(3) -> 4
4>>> from sage.all import *>>> class Foo():...
def f(self, *args):...
return self.x*self.x....:>>> foo = Foo()>>> from sage.doctest.fixtures import AttributeAccessTracerHelper>>> pat = AttributeAccessTracerHelper(foo)>>> pat.set("x", Integer(2))
write x = 2>>> pat.get("x")
read x = 22>>> pat.get("f")(Integer(3))
call f(3) -> 44

get(name)
Read an attribute from the wrapped delegate object.
If that value is a method (i.e. a callable object which is not contained in the dictionary of the object itself but
instead inherited from some class) then it is replaced by a wrapper function to report arguments and return
150

Chapter 9. Fixtures to help testing functionality

Sage's Doctesting Framework, Release 10.4

value. Otherwise an attribute read access is reported.
EXAMPLES:
sage: class Foo():....:def f(self, *args):....:
return self.x*self.x....:
sage: foo = Foo()
sage: foo.x = 2
sage: from sage.doctest.fixtures import AttributeAccessTracerHelper
sage: pat = AttributeAccessTracerHelper(foo)sage: pat.get("x")
read x = 2
2
sage: pat.get("f")(3)call f(3) -> 4
4>>> from sage.all import *>>> class Foo():...
def f(self, *args):...
return self.x*self.x....:>>> foo = Foo()>>> foo.x = Integer(2)>>> from sage.doctest.fixtures import AttributeAccessTracerHelper>>> pat = AttributeAccessTracerHelper(foo)>>> pat.get("x")
read x = 22>>> pat.get("f")(Integer(3))
call f(3) -> 4
4

set(name, val)Write an attribute to the wrapped delegate object.
The name and new value are also reported in the output.
EXAMPLES:
sage: class Foo():
....:
pass
....:
sage: foo = Foo()
sage: from sage.doctest.fixtures import AttributeAccessTracerHelper
sage: pat = AttributeAccessTracerHelper(foo)sage: pat.set("x", 2)
write x = 2
sage: foo.x
2>>> from sage.all import *>>> class Foo():
...
pass
....:
>>> foo = Foo()(continues on next page)

151

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> from sage.doctest.fixtures import AttributeAccessTracerHelper>>> pat = AttributeAccessTracerHelper(foo)>>> pat.set("x", Integer(2))
write x = 2>>> foo.x
2class sage.doctest.fixtures.AttributeAccessTracerProxy(delegate, **kwds)
Bases: object
Proxy object which prints all attribute and method access to an object.
The implementation is kept lean since all access to attributes of the proxy itself requires complicated syntax. For
this reason, the actual handling of attribute access is delegated to a AttributeAccessTracerHelper.
INPUT:• delegate – the actual object to be proxied.
• prefix – (default: " ") string to prepend to each printed output.
• reads – (default: True) whether to trace read access as well.
EXAMPLES:
sage: class Foo():....:def f(self, *args):....:
return self.x*self.x....:
sage: foo = Foo()
sage: from sage.doctest.fixtures import AttributeAccessTracerProxysage: pat = AttributeAccessTracerProxy(foo)sage: pat.x = 2
write x = 2
sage: pat.x
read x = 2
2
sage: pat.f(3)call f(3) -> 4
4>>> from sage.all import *>>> class Foo():...
def f(self, *args):...
return self.x*self.x....:>>> foo = Foo()>>> from sage.doctest.fixtures import AttributeAccessTracerProxy>>> pat = AttributeAccessTracerProxy(foo)>>> pat.x = Integer(2)write x = 2>>> pat.x
read x = 22>>> pat.f(Integer(3))
call f(3) -> 44

152

Chapter 9. Fixtures to help testing functionality

Sage's Doctesting Framework, Release 10.4__getattribute__(name)Read an attribute from the wrapped delegate object.
If that value is a method (i.e. a callable object which is not contained in the dictionary of the object itself but
instead inherited from some class) then it is replaced by a wrapper function to report arguments and return
value. Otherwise an attribute read access is reported.
EXAMPLES:
sage: class Foo():....:def f(self, *args):....:
return self.x*self.x....:
sage: foo = Foo()
sage: foo.x = 2
sage: from sage.doctest.fixtures import AttributeAccessTracerProxysage: pat = AttributeAccessTracerProxy(foo)sage: pat.x
read x = 2
2
sage: pat.f(3)call f(3) -> 4
4>>> from sage.all import *>>> class Foo():...
def f(self, *args):...
return self.x*self.x....:>>> foo = Foo()>>> foo.x = Integer(2)>>> from sage.doctest.fixtures import AttributeAccessTracerProxy>>> pat = AttributeAccessTracerProxy(foo)>>> pat.x
read x = 22>>> pat.f(Integer(3))
call f(3) -> 44__setattr__(name, val)Write an attribute to the wrapped delegate object.
The name and new value are also reported in the output.
EXAMPLES:
sage: class Foo():
....:
pass
....:
sage: foo = Foo()
sage: from sage.doctest.fixtures import AttributeAccessTracerProxysage: pat = AttributeAccessTracerProxy(foo)sage: pat.x = 2
write x = 2
sage: foo.x
2153

Sage's Doctesting Framework, Release 10.4>>> from sage.all import *>>> class Foo():
...
pass
....:
>>> foo = Foo()>>> from sage.doctest.fixtures import AttributeAccessTracerProxy>>> pat = AttributeAccessTracerProxy(foo)>>> pat.x = Integer(2)write x = 2>>> foo.x
2

sage.doctest.fixtures.reproducible_repr(val)String representation of an object in a reproducible way.
This tries to ensure that the returned string does not depend on factors outside the control of the doctest. One
example is the order of elements in a hash-based structure. For most objects, this is simply the repr of the object.
All types for which special handling had been implemented are covered by the examples below. If a doctest requires
special handling for additional types, this function may be extended appropriately. It is an error if an argument tothis function has a non-reproducible repr implementation and is not explicitly mentioned in an example casebelow.
INPUT:• val – an object to be represented
OUTPUT:A string representation of that object, similar to what repr returns but for certain cases with more guarantees toensure exactly the same result for semantically equivalent objects.
EXAMPLES:
sage: from sage.doctest.fixtures import reproducible_repr
sage: print(reproducible_repr(set(["a", "c", "b", "d"])))
set([ a , b , c , d ])sage: print(reproducible_repr(frozenset(["a", "c", "b", "d"])))
frozenset([ a , b , c , d ])sage: print(reproducible_repr([1, frozenset("cab"), set("bar"), 0]))[1, frozenset([ a , b , c ]), set([ a , b , r ]), 0]
sage: print(reproducible_repr({3.0: "three", "2": "two", 1: "one"}))˓→optional - sage.rings.real_mpfr{ 2 : two , 1: one , 3.00000000000000: three }sage: print(reproducible_repr("foo\nbar")) # demonstrate default casefoo\nbar#␣>>> from sage.all import *>>> from sage.doctest.fixtures import reproducible_repr>>> print(reproducible_repr(set(["a", "c", "b", "d"])))
set([ a , b , c , d ])>>> print(reproducible_repr(frozenset(["a", "c", "b", "d"])))
frozenset([ a , b , c , d ])>>> print(reproducible_repr([Integer(1), frozenset("cab"), set("bar"),␣
˓→Integer(0)]))[1, frozenset([ a , b , c ]), set([ a , b , r ]), 0]>>> print(reproducible_repr({RealNumber( 3.0 ): "three", "2": "two", Integer(1):˓→"one"}))# optional - sage.rings.real_mpfr{ 2 : two , 1: one , 3.00000000000000: three }(continues on next page)

154

Chapter 9. Fixtures to help testing functionality

Sage's Doctesting Framework, Release 10.4

(continued from previous page)>>> print(reproducible_repr("foo\nbar"))foo\nbar# demonstrate default casesage.doctest.fixtures.trace_method(obj, meth, **kwds)
Trace the doings of a given method. It prints method entry with arguments, access to members and other methods
during method execution as well as method exit with return value.
INPUT:• obj – the object containing the method.
• meth – the name of the method to be traced.
• prefix – (default: " ") string to prepend to each printed output.
• reads – (default: True) whether to trace read access as well.
EXAMPLES:
sage: class Foo():....:def f(self, arg=None):....:self.y = self.g(self.x)....:
if arg: return arg*arg....:
def g(self, arg):....:
return arg + 1....:
sage: foo = Foo()
sage: foo.x = 3
sage: from sage.doctest.fixtures import trace_method
sage: trace_method(foo, "f")sage: foo.f()enter f()read x = 3
call g(3) -> 4
write y = 4
exit f -> Nonesage: foo.f(3)enter f(3)read x = 3
call g(3) -> 4
write y = 4
exit f -> 99>>> from sage.all import *>>> class Foo():...
def f(self, arg=None):...
self.y = self.g(self.x)...
if arg: return arg*arg...
def g(self, arg):...
return arg + Integer(1)....:>>> foo = Foo()>>> foo.x = Integer(3)>>> from sage.doctest.fixtures import trace_method>>> trace_method(foo, "f")>>> foo.f()enter f()(continues on next page)

155

Sage's Doctesting Framework, Release 10.4

(continued from previous page)

read x = 3
call g(3) -> 4
write y = 4
exit f -> None>>> foo.f(Integer(3))
enter f(3)read x = 3
call g(3) -> 4
write y = 4
exit f -> 99156

Chapter 9. Fixtures to help testing functionality

CHAPTERTENINDICES AND TABLES• Index• Module Index• Search Page

157

Sage's Doctesting Framework, Release 10.4

158

Chapter 10. Indices and Tables

PYTHON MODULE INDEXdsage.doctest.control, 1
sage.doctest.external, 111
sage.doctest.fixtures, 149
sage.doctest.forker, 39
sage.doctest.parsing, 69
sage.doctest.reporting, 99
sage.doctest.sources, 21
sage.doctest.test, 121
sage.doctest.util, 143

159

Sage's Doctesting Framework, Release 10.4

160

Python Module Index

INDEX

Non-alphabeticalD__getattribute__()(sage.doctest.fixtures.At- detectable() (sage.doctest.external.AvailableSoftware
tributeAccessTracerProxy method), 152
method), 112__setattr__()(sage.doctest.fixtures.AttributeAc- dict_difference() (in module sage.doctest.util), 147
cessTracerProxy method), 153
DictAsObject (class in sage.doctest.sources), 21
dispatch()(sage.doctest.forker.DocTestDispatcherA
method), 39
abs_tol (sage.doctest.parsing.MarkedOutput attribute), do_fixup() (sage.doctest.parsing.SageOutputChecker
method), 83
69
add_files() (sage.doctest.control.DocTestController DocTestController (class in sage.doctest.control), 1
DocTestDefaults (class in sage.doctest.control), 17
method), 1
add_tolerance()(sage.doctest.parsing.SageOut- DocTestDispatcher (class in sage.doctest.forker), 39
DocTestReporter (class in sage.doctest.reporting), 99
putChecker method), 78
DocTestSource (class in sage.doctest.sources), 21
annotate() (sage.doctest.util.Timer method), 146attempted (sage.doctest.forker.TestResults attribute), 65 DocTestTask (class in sage.doctest.forker), 43
AttributeAccessTracerHelper(classin DocTestWorker (class in sage.doctest.forker), 44
dummy_handler() (in module sage.doctest.forker), 65
sage.doctest.fixtures), 150
AttributeAccessTracerProxy(classinE
sage.doctest.fixtures), 152
AvailableSoftware (class in sage.doctest.external), ending_docstring() (sage.doctest.sources.Python111
Source method), 26
ending_docstring()(sage.doctest.sources.RestB
Source method), 28(sage.doctest.sources.Texbasename()(sage.doctest.sources.FileDocTestSource ending_docstring()
Source method), 34
method), 22
environment variable
TERM, 121C
expand_files_into_sources()check_output()(sage.doctest.parsing.SageOut(sage.doctest.control.DocTestController method),
putChecker method), 79
3
cleanup()(sage.doctest.control.DocTestController
external_features()(in module sage.doctest.extermethod), 2
nal),
113
compile_and_execute() (sage.doctest.forker.SageDocTestRunner method), 48
copy() (sage.doctest.util.RecordingDict method), 144count_noun() (in module sage.doctest.util), 147
create_doctests()(sage.doctest.sources.FileDocTestSource method), 23
create_doctests()(sage.doctest.sources.StringDocTestSource method), 33
create_run_id() (sage.doctest.control.DocTestController method), 3

F
failed (sage.doctest.forker.TestResults attribute), 65
file_optional_tags(sage.doctest.parsing.SageDocTestParser attribute), 71
file_optional_tags()(sage.doctest.sources.DocTestSource method), 21
file_optional_tags() (sage.doctest.sources.FileDocTestSource method), 23
FileDocTestSource (class in sage.doctest.sources), 22
161

Sage's Doctesting Framework, Release 10.4
filter_sources() (sage.doctest.control.DocTestCon- K
troller method), 5kill() (sage.doctest.forker.DocTestWorker method), 45
finalize()(sage.doctest.reporting.DocTestReporter
method), 99
Lflush() (sage.doctest.control.Logger method), 18
load_baseline_stats()(sage.doctest.control.DocTestController method), 5G
load_environment() (sage.doctest.control.DocTestget() (sage.doctest.fixtures.AttributeAccessTracerHelper
Controller method), 6
method), 150
load_stats() (sage.doctest.control.DocTestController
get() (sage.doctest.util.RecordingDict method), 145
method), 7
get_basename() (in module sage.doctest.sources), 37 log() (sage.doctest.control.DocTestController method), 7
get_source() (in module sage.doctest.parsing), 87
Logger (class in sage.doctest.control), 17
getvalue()(sage.doctest.forker.SageSpoofInOut long (sage.doctest.parsing.SageDocTestParser attribute),
method), 62
71

H

M

has_4ti2() (in module sage.doctest.external), 113
has_cplex() (in module sage.doctest.external), 114
has_dvipng() (in module sage.doctest.external), 114
has_ffmpeg() (in module sage.doctest.external), 114
has_graphviz() (in module sage.doctest.external), 114
has_gurobi() (in module sage.doctest.external), 115
has_imagemagick() (in module sage.doctest.external), 115
has_internet() (in module sage.doctest.external), 115
has_latex() (in module sage.doctest.external), 115
has_lualatex() (in module sage.doctest.external), 116
has_macaulay2() (in module sage.doctest.external),
116
has_magma() (in module sage.doctest.external), 116
has_maple() (in module sage.doctest.external), 116
has_mathematica() (in module sage.doctest.external), 117
has_matlab() (in module sage.doctest.external), 117
has_octave() (in module sage.doctest.external), 117
has_pandoc() (in module sage.doctest.external), 117
has_pdf2svg() (in module sage.doctest.external), 118
has_pdflatex() (in module sage.doctest.external), 118
has_rubiks() (in module sage.doctest.external), 118
has_scilab() (in module sage.doctest.external), 118
has_xelatex() (in module sage.doctest.external), 119
hidden()(sage.doctest.external.AvailableSoftware
method), 112
human_readable_escape_sequences()(sage.doctest.parsing.SageOutputChecker
method), 85

make_marked_output()(inmodulesage.doctest.parsing), 88
make_recording_dict()(inmodulesage.doctest.util), 148MarkedOutput (class in sage.doctest.parsing), 69
modulesage.doctest.control, 1
sage.doctest.external, 111
sage.doctest.fixtures, 149
sage.doctest.forker, 39
sage.doctest.parsing, 69
sage.doctest.reporting, 99
sage.doctest.sources, 21
sage.doctest.test, 121
sage.doctest.util, 143

I

N
NestedName (class in sage.doctest.util), 143O
optional_only(sage.doctest.parsing.SageDocTestParser attribute), 71
optional_tags(sage.doctest.parsing.SageDocTestParser attribute), 71
optionals (sage.doctest.parsing.SageDocTestParser attribute), 71
OriginalSource (class in sage.doctest.parsing), 70
output_difference() (sage.doctest.parsing.SageOutputChecker method), 85

P

parallel_dispatch() (sage.doctest.forker.DocTestin_lib()(sage.doctest.sources.FileDocTestSource
Dispatcher method), 40
method), 24
parse()(sage.doctest.parsing.SageDocTestParser
init_sage() (in module sage.doctest.forker), 65
method), 71
issuperset() (sage.doctest.external.AvailableSoftwareparse_docstring() (sage.doctest.sources.RestSource
method), 113
method), 29
162

Index

Sage's Doctesting Framework, Release 10.4

parse_docstring() (sage.doctest.sources.SourceLanguage method), 31
parse_file_optional_tags()(inmodulesage.doctest.parsing), 88
parse_optional_tags()(inmodulesage.doctest.parsing), 89
parse_tolerance() (in module sage.doctest.parsing),
92
pre_hash() (in module sage.doctest.parsing), 92
printpath() (sage.doctest.sources.FileDocTestSource
method), 25
probed_tags (sage.doctest.parsing.SageDocTestParser
attribute), 77
PythonSource (class in sage.doctest.sources), 25

R
random (sage.doctest.parsing.MarkedOutput attribute), 69
read_messages() (sage.doctest.forker.DocTestWorker
method), 46
RecordingDict (class in sage.doctest.util), 143
reduce_hex() (in module sage.doctest.parsing), 93
rel_tol (sage.doctest.parsing.MarkedOutput attribute),
69
report()(sage.doctest.reporting.DocTestReporter
method), 102
report_failure()(sage.doctest.forker.SageDocTestRunner method), 51
report_head()(sage.doctest.reporting.DocTestReporter method), 108
report_overtime()(sage.doctest.forker.SageDocTestRunner method), 53
report_start()(sage.doctest.forker.SageDocTestRunner method), 54
report_success()(sage.doctest.forker.SageDocTestRunner method), 55
report_unexpected_exception()(sage.doctest.forker.SageDocTestRunnermethod), 56
reproducible_repr() (in module sage.doctest.fixtures), 154
RestSource (class in sage.doctest.sources), 28
RIFtol() (in module sage.doctest.parsing), 71
run() (sage.doctest.control.DocTestController method), 9
run() (sage.doctest.forker.DocTestWorker method), 47
run() (sage.doctest.forker.SageDocTestRunner method),57
run_doctests() (in module sage.doctest.control), 18
run_doctests()(sage.doctest.control.DocTestController method), 13
run_val_gdb()(sage.doctest.control.DocTestController method), 14

S
sage.doctest.control
Index

module, 1
sage.doctest.external
module, 111
sage.doctest.fixturesmodule, 149
sage.doctest.forker
module, 39
SageDocTestParser (class in sage.doctest.parsing),
71
sage.doctest.parsing
module, 69
sage.doctest.reporting
module, 99
SageDocTestRunner (class in sage.doctest.forker), 48
sage.doctest.sources
module, 21
sage.doctest.test
module, 121
sage.doctest.util
module, 143
SageOutputChecker (class in sage.doctest.parsing),
77
SageSpoofInOut (class in sage.doctest.forker), 61
save_result_output()(sage.doctest.forker.DocTestWorker method), 47
save_stats() (sage.doctest.control.DocTestController
method), 14
second_on_modern_computer()(sage.doctest.control.DocTestController method),15
seen() (sage.doctest.external.AvailableSoftware method),113
serial_dispatch() (sage.doctest.forker.DocTestDispatcher method), 42
set() (sage.doctest.fixtures.AttributeAccessTracerHelper
method), 151
showwarning_with_traceback() (in module
sage.doctest.forker), 67
signal_name() (in module sage.doctest.reporting), 110
skipdir() (in module sage.doctest.control), 19
skipfile() (in module sage.doctest.control), 19
sort_sources()(sage.doctest.control.DocTestController method), 15
source_baseline() (sage.doctest.control.DocTestController method), 16
SourceLanguage (class in sage.doctest.sources), 31
start() (sage.doctest.forker.DocTestWorker method), 48
start() (sage.doctest.util.RecordingDict method), 145
start() (sage.doctest.util.Timer method), 146
start_finish_can_overlap(sage.doctest.sources.PythonSource attribute), 26
start_finish_can_overlap(sage.doctest.sources.RestSource
attribute),
30163Sage's Doctesting Framework, Release 10.4

start_finish_can_overlap(sage.doctest.sources.TexSource
attribute),
35
start_spoofing() (sage.doctest.forker.SageSpoofInOut method), 63
starting_docstring()(sage.doctest.sources.PythonSource
method),26
starting_docstring() (sage.doctest.sources.RestSource method), 30
starting_docstring() (sage.doctest.sources.TexSource method), 35
stop() (sage.doctest.util.Timer method), 146
stop_spoofing()(sage.doctest.forker.SageSpoofInOut method), 64
StringDocTestSource(classin
sage.doctest.sources), 32
summarize() (sage.doctest.forker.SageDocTestRunnermethod), 58

T
TERM, 121
TestResults (class in sage.doctest.forker), 65
TexSource (class in sage.doctest.sources), 34
Timer (class in sage.doctest.util), 146
tol (sage.doctest.parsing.MarkedOutput attribute), 69
trace_method() (in module sage.doctest.fixtures), 155Uunparse_optional_tags()(inmodulesage.doctest.parsing), 93
update() (sage.doctest.parsing.MarkedOutput method),
70
update_digests()(sage.doctest.forker.SageDocTestRunner method), 59
update_optional_tags()(inmodulesage.doctest.parsing), 94
update_results()(sage.doctest.forker.SageDocTestRunner method), 60

W
were_doctests_with_optional_tag_run()(sage.doctest.reporting.DocTestReporter method),109
write() (sage.doctest.control.Logger method), 18

164

Index