Package: fault
FaultA Python package for testing hardware (part of the magma ecosystem).API DocumentationCHANGELOGInstallationpip install faultDocumentationCheck out thefault tutorialActionsTesterIntegrating External VerilogPropertiesSupported SimulatorsDigital simulationVerilatorIcarus VerilogCadence Incisive/XceliumSynopsys VCSXilinx VivadoFormal verificationEngines supported bypySMTAnalog simulationngspiceCadence SpectreSynopsys HSPICEMixed-signal simulationVerilog-AMSvia Cadence Incisive/XceliumExampleHere is a simple ALU defined in magma.importmagmaasmclassConfigReg(m.Circuit):io=m.IO(D=m.In(m.Bits[2]),Q=m.Out(m.Bits[2]))+\m.ClockIO(has_ce=True)reg=m.Register(m.Bits[2],has_enable=True)(name="conf_reg")io.Q@=reg(io.D,CE=io.CE)classSimpleALU(m.Circuit):io=m.IO(a=m.In(m.UInt[16]),b=m.In(m.UInt[16]),c=m.Out(m.UInt[16]),config_data=m.In(m.Bits[2]),config_en=m.In(m.Enable))+m.ClockIO()opcode=ConfigReg(name="config_reg")(io.config_data,CE=io.config_en)io.c@=m.mux([io.a+io.b,io.a-io.b,io.a*io.b,io.a^io.b],opcode)Here's an example test in fault that uses the configuration interface, expects
a value on the internal register, and checks the result of performing the
expected operation.importoperatorimportfaultops=[operator.add,operator.sub,operator.mul,operator.floordiv]tester=fault.Tester(SimpleALU,SimpleALU.CLK)tester.circuit.CLK=0tester.circuit.config_en=1foriinrange(0,4):tester.circuit.config_data=itester.step(2)tester.circuit.a=3tester.circuit.b=2tester.eval()tester.circuit.c.expect(ops[i](3,2))We can run this with three different simulatorstester.compile_and_run("verilator",flags=["-Wno-fatal"],directory="build")tester.compile_and_run("system-verilog",simulator="ncsim",directory="build")tester.compile_and_run("system-verilog",simulator="vcs",directory="build")Working with internal signalsFault supports peeking, expecting, and printing internal signals. For theverilatortarget, you should use the keyword argumentmagma_optswith"verilator_debug"set to true.  This will cause coreir to compile the verilog
with the required debug comments.  Example:tester.compile_and_run("verilator",flags=["-Wno-fatal"],magma_opts={"verilator_debug":True},directory="build")If you're usingmantle.Registerfrom thecoreirimplementation, you can
also poke the internal register value directly using thevaluefield.  Notice
thatconf_regis defined inConfigRegto be an instance ofmantle.Registerand the test bench pokes it by settingconfg_reg.valueequal to1.tester=fault.Tester(SimpleALU,SimpleALU.CLK)tester.circuit.CLK=0# Set config_en to 0 so stepping the clock doesn't clobber the poked valuetester.circuit.config_en=0# Initializetester.step(2)foriinreversed(range(4)):tester.circuit.config_reg.conf_reg.value=itester.step(2)tester.circuit.config_reg.conf_reg.O.expect(i)# You can also print these internal signals using the getattr interfacetester.print("O=%d\n",tester.circuit.config_reg.conf_reg.O)FAQHow can I write test bench logic that depends on the runtime state of the circuit?A common pattern in testing is to only perform certain actions depending on the
state of the circuit.  For example, one may only want to expect an output value
when a valid signal is high, ignoring it otherwise.  Another pattern is to change
the expected value over time by using a looping structure.  Finally, one may
want to expect a value that is a function of other runtime values.  To support
these pattern,faultprovides support "peeking" values, performing expressions
on "peeked" values, if statements, and while loops.Peek ExpressionsSuppose we had a circuit as follows:classBinaryOpCircuit(m.Circuit):io=m.IO(I0=m.In(m.UInt[5]),I1=m.In(m.UInt[5]),O=m.Out(m.UInt[5]))io.O@=io.I0+io.I1&(io.I1-io.I0)We can write a generic test that expects the outputOin terms
of the inputsI0andI1(rather than computing the expected value in
Python).tester=fault.Tester(BinaryOpCircuit)for_inrange(5):tester.poke(tester._circuit.I0,hwtypes.BitVector.random(5))tester.poke(tester._circuit.I1,hwtypes.BitVector.random(5))tester.eval()expected=tester.circuit.I0+tester.circuit.I1expected&=tester.circuit.I1-tester.circuit.I0tester.circuit.O.expect(expected)This is a useful pattern for writing reuseable test components (e.g.  composign
the output checking logic with various input stimuli generators).Control StructuresThetester._while(<test>)action accepts a Peek value or expression as the test condition for a loop and returns a child tester that allows the user to add actions to the body of the loop.  Here's a simple example that loops until a done signal is asserted, printing some debug information in the loop body:# Wait for loop to completeloop=tester._while(dut.n_done)debug_print(loop,dut)loop.step()loop.step()# check final statetester.circuit.count.expect(expected_num_cycles-1)Notice that you can also add actions after the loop to check expected behavior
after the loop has completed.Thetester._if(<test>)action behaves similarly by accepting a test peek value or expression and conditionally executes actions depending on the
result of the expression.  Here is a simple example:if_tester=tester._if(tester.circuit.O==0)if_tester.circuit.I=1else_tester=if_tester._else()else_tester.circuit.I=0tester.eval()Thetester._for(<num_iter>)action provides a simple way to write a loop over
a fixed number of iterations.  Use the attributeindexto get access to the
current iteration, for example:loop=tester._for(8)loop.poke(circ.I,loop.index)loop.eval()tester.expect(circ.O,loop.index)What Python values can I use to poke/expect ports?Here are the supported Python values for poking the following port types:m.Bit-bool(True/False) orint(0/1) orhwtypes.Bitm.Bits[N]-hwtypes.BitVector[N],int(where the number of bits used to
express it is equal toN)m.SInt[N]-hwtypes.SIntVector[N],int(where the number of bits used to
express it is equal toN)m.UInt[N]-hwtypes.UIntVector[N],int(where the number of bits used to
express it is equal toN)m.Array[N, T]-list(where the length of the list is equal toNand
the elements recursively conform to the supported types of values forT).
For example, suppose I have a portIof typem.Array[3, m.Bits[3]].
I can poke it as follows:val=[random.randint(0,(1<<4)-1)for_inrange(3)]tester.poke(circ.I,val)You can also poke it by element as follows:foriinrange(3):val=random.randint(0,(1<<4)-1)tester.poke(circ.I[i],val)tester.eval()tester.expect(circ.O[i],val)m.Tuple(a=m.Bits[4], b=m.Bits[4])-tuple(where the length of the tuple is equal to the number of fields),dict(where there is a one-to-one mapping between key/value pairs to the tuple fields).  Example:tester.circuit.I=(4,2)tester.eval()tester.circuit.O.expect((4,2))tester.circuit.I={"a":4,"b":2}tester.eval()tester.circuit.O.expect({"a":4,"b":2})How do I generate waveforms with fault?Fault supports generating.vcddumps when using theverilatorandsystem-verilog/ncsimtarget.For theverilatortarget, use theflagskeyword argument to pass the--traceflag.  For example,tester.compile_and_run("verilator", flags=["-Wno-fatal", "--trace"])The--traceflag must be passed through to verilator so it generates code
that supports waveform dumping. The test harness generated by fault will
include the required logic for invokingtracer->dump(main_time)for every
call toevalandstep.main_timeis incremented for every call to step.
The output.vcdfile will be saved in the filelogs/{circuit_name}wherecircuit_nameis the name of the ciruit passed toTester.  Thelogsdirectory will be placed in the same directory as the generated harness, which
is controlled by thedirectorykeyword argument (by default this is"build/").For thesystem-verilogtarget, enable this feature using thecompile_and_runparameterdump_waveform=True.  By default, the waveform
file will be namedwaveforms.vcdforncsimandwaveforms.vpdforvcs.
The name of the file can be changed using the parameterwaveform_file="<file_name>".Thevcssimulator also supports dumpingfsdbby using the argumentwaveform_type="fsdb".  For this to work, you'll need to also use theflagsargument using the path defined in your verdi manual.  For example,$VERDI_HOME/doc/linking_dumping.pdf.Here is an example using an older version of verdi (using the VERDIHOME
environment variable):verdi_home=os.environ["VERDIHOME"]# You may need to change the 'vcs_latest' and 'LINUX64' parts of the path# depending on your verdi version, please consult# $VERDI_HOME/doc/linking_dumping.pdfflags=['-P',f'{verdi_home}/share/PLI/vcs_latest/LINUX64/novas.tab',f'{verdi_home}/share/PLI/vcs_latest/LINUX64/pli.a']tester.compile_and_run(target="system-verilog",simulator="vcs",waveform_type="fsdb",dump_waveforms=True,flags=flags)Here's an example for a newer version of verdiverdi_home=os.environ["VERDI_HOME"]flags=['-P',f'{verdi_home}/share/PLI/VCS/linux64/novas.tab',f'{verdi_home}/share/PLI/VCS/linux64/pli.a']tester.compile_and_run(target="system-verilog",simulator="vcs",waveform_type="fsdb",dump_waveforms=True,flags=flags)To configure fsdb dumping, use thefsdb_dumpvars_argsparameter of the
compile_and_run command to pass a string to the$fsdbDumpvars()function.For example:tester.compile_and_run(target="system-verilog",simulator="vcs",waveform_type="fsdb",dump_waveforms=True,fsdb_dumpvars_args='0, "dut"')will produce:$fsdbDumpvars(0,"dut");inside the generated test bench.How do I pass through flags to the simulator?Theverilatorandsystem-verilogtarget support the parameterflagswhich
accepts a list of flags (strings) that will be passed through to the simulator
command (verilatorfor verilator,irunfor ncsim,vcsfor vcs, andiverilogfor iverilog).Can I include a message to print when an expect fails?Use themsgargument to the expect action. You can either pass a standalone
string, e.g.tester.circuit.O.expect(0,msg="my error message")or you can pass a printf/$display style message using a tuple.  The first argument
should be the format string, the subsequent arguments are the format values,
e.g.tester.circuit.O.expect(0,msg=("MY_MESSAGE: got%x, expected 0!",tester.circuit.O))Can I display or print values from my testbench?Yes, you can use thetester.printAPI which accepts a format string and a
variable number of arguments.  Here's an example:tester=fault.Tester(circ,circ.CLK)tester.poke(circ.I,0)tester.eval()tester.expect(circ.O,0)tester.poke(circ.CLK,0)tester.step()tester.print("%08x\n",circ.O)Can I just generate a test bench without running it?Yes, here's an example:# compile the testertester.compile("verilator")# generate the test bench file (returns the name of the file)tb_file=tester.generate_test_bench("verilator")or for system verilogtester.compile("system-verilog",simulator="ncsim")tb_file=tester.generate_test_bench("system-verilog")Using the ReadyValid TesterFault provides aReadyValidTesterthat provides a few convenient features for
unit testingReadyValidinterfaces with sequences.Consider the following circuit:classMain2(m.Circuit):io=m.IO(I=m.Consumer(m.ReadyValid[m.UInt[8]]),O=m.Producer(m.ReadyValid[m.UInt[8]]),inc=m.In(m.UInt[8]),)+m.ClockIO()count=m.Register(m.UInt[2])()count.I@=count.O+1enable=io.I.valid&(count.O==3)&io.O.readyio.I.ready@=enableio.O.data@=m.Register(m.UInt[8],has_enable=True)()(io.I.data+io.inc,CE=enable)io.O.valid@=enableThe output streamOis the input streamIincremented by the value ofincand delayed by 4 cycles.Here's a simple test that provides the input sequenceIand expected output
sequenceOdeftest_lifted_ready_valid_sequence_simple():I=[BitVector.random(8)for_inrange(8)]+[0]O=[0]+[i+2foriinI[:-1]]tester=f.ReadyValidTester(Main2,{"I":I,"O":O})tester.circuit.inc=2tester.finish_sequences()tester.compile_and_run("verilator",disp_type="realtime")Notice that we provide the sequences as a dictionary mapping port name to
sequence in the constructor.  Afterwards, we are free topokevalues like in a normal tester, in this case provide2forincwhich will satisfy the provided stream.NOTE:The user must explicitly use thetester.circuitpeek/poke
interface, or calltester.poke(tester._circuit, value)since the user circuit
is wrapped internally (cannot calltester.poke(Main2, value)).The test finishes by callingtester.finish_sequences()which is a convenience
API that waits for the provided sequences to finish.Here's a different version of the above test that should fail (changing theincvalue mid test).deftest_lifted_ready_valid_sequence_simple_fail():I=[BitVector.random(8)for_inrange(8)]+[0]O=[0]+[i+2foriinI[:-1]]tester=f.ReadyValidTester(Main2,{"I":I,"O":O})tester.circuit.inc=2# Should work for a few cyclesforiinrange(9):tester.advance_cycle()# Bad inc should failtester.circuit.inc=3tester.finish_sequences()withpytest.raises(AssertionError):tester.compile_and_run("verilator",disp_type="realtime")Finally, here's a variant that changes theincvalue over time to match the expected sequence.deftest_lifted_ready_valid_sequence_changing_inc():I=[BitVector.random(8)for_inrange(8)]+[0]O=[0]+[I[i]+((i+1)%2)foriinrange(8)]tester=f.ReadyValidTester(Main2,{"I":I,"O":O})# Sequence expects inc to change over timeforiinrange(8):tester.circuit.inc=i%2tester.advance_cycle()tester.wait_until_high(tester.circuit.O.ready&tester.circuit.O.valid)# Advance one cycle to finish last handshaketester.advance_cycle()tester.expect_sequences_finished()tester.compile_and_run("verilator",disp_type="realtime")NOTE:At the end of the test, we callexpect_sequences_finished()to
assert that the sequences have all been processed, otherwise it's possible tha
the test could pass without completing the sequences.