Package: tinyfive
TinyFiveTinyFive is a lightweight RISC-V emulator and assembler written entirely in Python:Useful for running neural networks on RISC-V: Simulate your RISC-V assembly code along with a neural network in Keras or PyTorch (and without relying on RISC-V toolchains).Custom instructions can be added for easy HW/SW codesign in Python (without C++ and compiler toolchains).If you want to learn how RISC-V works, TinyFive lets you play with instructions and assembly code inthis colab.TinyFive might also be useful for ML scientists who are using ML/RL for compiler optimizations (see e.g.CompilerGym) or to replace compiler toolchains by AI.Can be very fast if you only use the upper-case instructions defined in thefirst ~200 lines of machine.py.Fewer than 1000 linesof code (w/o tests and examples)Uses NumPy for mathContentsInstallationUsageExample 1: Multiply two numbersExample 2: Add two vectorsExample 3: Multiply two matricesExample 4: Neural network layersExample 5: MobileNetRunning in colabRunning without packageContributeLatest statusSpeedComparisonReferencesTiny Tech promiseInstallationpip install tinyfiveUsageTinyFive can be used in the following three ways:Option A:Use upper-case instructions such asADD()andMUL(), see examples 1.1, 1.2, 2.1, and 3.1 below.Option B:Useasm()andexe()functions without branch instructions, see examples 1.3 and 2.2 below.Option C:Useasm()andexe()functions with branch instructions, see example 2.3, 3.2, and 3.3 below.For the examples below, import and instantiate a RISC-V machine with at least 4KB of memory as follows:fromtinyfive.machineimportmachinem=machine(mem_size=4000)# instantiate RISC-V machine with 4KB of memoryExample 1: Multiply two numbersExample 1.1:Use upper-case instructions (option A) with back-door loading of registers.m.x[11]=6# manually load '6' into register x[11]m.x[12]=7# manually load '7' into register x[12]m.MUL(10,11,12)# x[10] := x[11] * x[12]print(m.x[10])# Output: 42Example 1.2:Same as example 1.1, but now load the data from memory. Specifically, the data values are stored at addresses 0 and 4. Here, each value is 32 bits wide (i.e. 4 bytes wide), which occupies 4 addresses in the byte-wide memory.m.write_i32(6,0)# manually write '6' into mem[0] (memory @ address 0)m.write_i32(7,4)# manually write '7' into mem[4] (memory @ address 4)m.LW(11,0,0)# load register x[11] from mem[0 + 0]m.LW(12,4,0)# load register x[12] from mem[4 + 0]m.MUL(10,11,12)# x[10] := x[11] * x[12]print(m.x[10])# Output: 42Example 1.3:Same as example 1.2, but now useasm()andexe()(option B). The assembler functionasm()function takes an instruction and converts it into machine code and stores it in memory at addresss.pc. Once the entire assembly program is written into memorymem[], theexe()function (aka ISS) can then exectute the machine code stored in memory.m.write_i32(6,0)# manually write '6' into mem[0] (memory @ address 0)m.write_i32(7,4)# manually write '7' into mem[4] (memory @ address 4)# store assembly program in mem[] starting at address 4*20m.pc=4*20m.asm('lw',11,0,0)# load register x[11] from mem[0 + 0]m.asm('lw',12,4,0)# load register x[12] from mem[4 + 0]m.asm('mul',10,11,12)# x[10] := x[11] * x[12]# execute program from address 4*20: execute 3 instructions and then stopm.exe(start=4*20,instructions=3)print(m.x[10])# Output: 42Example 2: Add two vectorsWe are using the following memory map for adding two 8-element vectorsres[] := a[] + b[], where each vector element is 32 bits wide (i.e. each element occupies 4 byte-addresses in memory).Byte addressContents0   .. 4*7a-vector:a[0]is at address 0,a[7]is at address 4*74*8  .. 4*15b-vector:b[0]is at address 4*8,b[7]is at address 4*154*16 .. 4*23result-vector:res[0]is at address 4*16,res[7]is at address 4*23Example 2.1:Use upper-case instructions (option A) with Python for-loop.# generate 8-element vectors a[] and b[] and store them in memorya=np.random.randint(100,size=8)b=np.random.randint(100,size=8)m.write_i32_vec(a,0)# write vector a[] to mem[0]m.write_i32_vec(b,4*8)# write vector b[] to mem[4*8]# pseudo-assembly for adding vectors a[] and b[] using Python for-loopforiinrange(8):m.LW(11,4*i,0)# load x[11] with a[i] from mem[4*i + 0]m.LW(12,4*(i+8),0)# load x[12] with b[i] from mem[4*(i+8) + 0]m.ADD(10,11,12)# x[10] := x[11] + x[12]m.SW(10,4*(i+16),0)# store results in mem[], starting at address 4*16# compare results against golden referenceres=m.read_i32_vec(4*16,size=8)# read result vector from address 4*16ref=a+b# golden reference: simply add a[] + b[]print(res-ref)# print difference (should be all-zero)# Output: [0 0 0 0 0 0 0 0]Example 2.2: Same as example 2.1, but now useasm()andexe()functions without branch instructions (option B).# generate 8-element vectors a[] and b[] and store them in memorya=np.random.randint(100,size=8)b=np.random.randint(100,size=8)m.write_i32_vec(a,0)# write vector a[] to mem[0]m.write_i32_vec(b,4*8)# write vector b[] to mem[4*8]# store assembly program in mem[] starting at address 4*48m.pc=4*48foriinrange(8):m.asm('lw',11,4*i,0)# load x[11] with a[i] from mem[4*i + 0]m.asm('lw',12,4*(i+8),0)# load x[12] with b[i] from mem[4*(i+8) + 0]m.asm('add',10,11,12)# x[10] := x[11] + x[12]m.asm('sw',10,4*(i+16),0)# store results in mem[], starting at address 4*16# execute program from address 4*48: execute 8*4 instructions and then stopm.exe(start=4*48,instructions=8*4)# compare results against golden referenceres=m.read_i32_vec(4*16,size=8)# read result vector from address 4*16ref=a+b# golden reference: simply add a[] + b[]print(res-ref)# print difference (should be all-zero)# Output: [0 0 0 0 0 0 0 0]Example 2.3:Same as example 2.2, but now useasm()andexe()functions with branch instructions (option C). Thelbl()function defines labels, which are symbolic names that represent memory addresses. These labels improve the readability of branch instructions and mark the start and end of the assembly code executed by theexe()function.# generate 8-element vectors a[] and b[] and store them in memorya=np.random.randint(100,size=8)b=np.random.randint(100,size=8)m.write_i32_vec(a,0)# write vector a[] to mem[0]m.write_i32_vec(b,4*8)# write vector b[] to mem[4*8]# store assembly program starting at address 4*48m.pc=4*48# x[13] is the loop-variable that is incremented by 4: 0, 4, .., 28# x[14] is the constant 28+4 = 32 for detecting the end of the for-loopm.lbl('start')# define label 'start'm.asm('add',13,0,0)# x[13] := x[0] + x[0] = 0 (because x[0] is always 0)m.asm('addi',14,0,32)# x[14] := x[0] + 32 = 32 (because x[0] is always 0)m.lbl('loop')# label 'loop'm.asm('lw',11,0,13)# load x[11] with a[] from mem[0 + x[13]]m.asm('lw',12,4*8,13)# load x[12] with b[] from mem[4*8 + x[13]]m.asm('add',10,11,12)# x[10] := x[11] + x[12]m.asm('sw',10,4*16,13)# store x[10] in mem[4*16 + x[13]]m.asm('addi',13,13,4)# x[13] := x[13] + 4 (increment x[13] by 4)m.asm('bne',13,14,'loop')# branch to 'loop' if x[13] != x[14]m.lbl('end')# label 'end'# execute program: start at label 'start', stop when label 'end' is reachedm.exe(start='start',end='end')# compare results against golden referenceres=m.read_i32_vec(4*16,size=8)# read result vector from address 4*16ref=a+b# golden reference: simply add a[] + b[]print(res-ref)# print difference (should be all-zero)# Output: [0 0 0 0 0 0 0 0]A slightly more efficient implementation would decrement the loop variablex[13](instead of incrementing) so that the branch instruction compares againstx[0] = 0(instead of the constant stored inx[14]), which frees up registerx[14]and reduces the total number of instructions by 1.Useprint_perf()to analyze performance anddump_state()to print out the current values of the register files and the the program counter (PC) as follows:>>>m.print_perf()Opscounters:{'total':50,'load':16,'store':8,'mul':0,'add':18,'madd':0,'branch':8}x[]regfile:5outof31x-registersareusedf[]regfile:0outof32f-registersareusedImagesize:32Bytes>>>m.dump_state()pc:224x[0]:0,x[1]:0,x[2]:0,x[3]:0x[4]:0,x[5]:0,x[6]:0,x[7]:0x[8]:0,x[9]:0,x[10]:34,x[11]:27x[12]:7,x[13]:32,x[14]:32,x[15]:0x[16]:0,x[17]:0,x[18]:0,x[19]:0x[20]:0,x[21]:0,x[22]:0,x[23]:0x[24]:0,x[25]:0,x[26]:0,x[27]:0x[28]:0,x[29]:0,x[30]:0,x[31]:0Example 3: Multiply two matricesWe are using the following memory map for multiplying two 4x4 matrices asres := np.matmul(A, B), where each matrix element is 32 bits wide (i.e. each element occupies 4 byte-addresses in memory).Byte addressContents0    .. 4*15A-matrix in row-major order:A[0, 0], A[0, 1], ... A[3, 3]4*16 .. 4*31B-matrix in row-major order:B[i, j]is at address4*(16+i*4+j)4*32 .. 4*47result matrixres[0, 0] ... res[3, 3]Example 3.1:Use upper-case instructions (option A) with Python for-loop.# generate 4x4 matrices A and B and store them in memoryA=np.random.randint(100,size=(4,4))B=np.random.randint(100,size=(4,4))m.write_i32_vec(A.flatten(),0)# write matrix A to mem[0]m.write_i32_vec(B.flatten(),4*16)# write matrix B to mem[4*16]# pseudo-assembly for matmul(A, B) using Python for-loopsforiinrange(4):# load x[10] ... x[13] with row i of Aforkinrange(4):m.LW(10+k,4*(4*i+k),0)# load x[10+k] with A[i, k]forjinrange(4):# calculate dot productm.LW(18,4*(16+j),0)# load x[18] with B[0, j]m.MUL(19,10,18)# x[19] := x[10] * x[18] = A[i, 0] * B[0, j]forkinrange(1,4):m.LW(18,4*(16+4*k+j),0)# load x[18] with B[k, j]m.MUL(18,10+k,18)# x[18] := x[10+k] * x[18] = A[i, k] * B[k, j]m.ADD(19,19,18)# x[19] := x[19] + x[18]m.SW(19,4*(32+i*4+j),0)# store res[i, j] from x[19]# compare results against golden referenceres=m.read_i32_vec(4*32,size=4*4).reshape(4,4)# read result matrixref=np.matmul(A,B)# golden referenceprint(np.array_equal(res,ref))# should return 'True'# Output: TrueExample 3.2:Same as example 3.1, but now useasm()andexe()functions with branch instructions (option C).# generate 4x4 matrices A and B and store them in memoryA=np.random.randint(100,size=(4,4))B=np.random.randint(100,size=(4,4))m.write_i32_vec(A.flatten(),0)# write matrix A to mem[0]m.write_i32_vec(B.flatten(),4*16)# write matrix B to mem[4*16]# store assembly program starting at address 4*128m.pc=4*128# here, we decrement the loop variables down to 0 so that we don't need an# additional register to hold the constant for detecting the end of the loop:#  - x[20] is 4*4*i (i.e. the outer-loop variable) and is decremented by 16 from 64#  - x[21] is 4*j (i.e. the inner-loop variable) and is decremented by 4 from 16m.lbl('start')m.asm('addi',20,0,64)# x[20] := 0 + 64m.lbl('outer-loop')m.asm('addi',20,20,-16)# decrement loop-variable: x[20] := x[20] - 16m.asm('lw',10,0,20)# load x[10] with A[i, 0] from mem[0 + x[20]]m.asm('lw',11,4,20)# load x[11] with A[i, 1] from mem[4 + x[20]]m.asm('lw',12,2*4,20)# load x[12] with A[i, 2] from mem[2*4 + x[20]]m.asm('lw',13,3*4,20)# load x[13] with A[i, 3] from mem[3*4 + x[20]]m.asm('addi',21,0,16)# reset loop-variable j: x[21] := 0 + 16m.lbl('inner-loop')m.asm('addi',21,21,-4)# decrement j: x[21] := x[21] - 4m.asm('lw',18,4*16,21)# load x[18] with B[0, j] from mem[4*16 + x[21]]m.asm('mul',19,10,18)# x[19] := x[10] * x[18] = A[i, 0] * B[0, j]m.asm('lw',18,4*(16+4),21)# load x[18] with B[1, j]m.asm('mul',18,11,18)# x[18] := x[11] * x[18] = A[i, 1] * B[1, j]m.asm('add',19,19,18)# x[19] := x[19] + x[18]m.asm('lw',18,4*(16+2*4),21)# load x[18] with B[2, j]m.asm('mul',18,12,18)# x[18] := x[11] * x[18] = A[i, 2] * B[2, j]m.asm('add',19,19,18)# x[19] := x[19] + x[18]m.asm('lw',18,4*(16+3*4),21)# load x[18] with B[3, j]m.asm('mul',18,13,18)# x[18] := x[11] * x[18] = A[i, 3] * B[3, j]m.asm('add',19,19,18)# x[19] := x[19] + x[18]m.asm('add',24,20,21)# calculate base address for result-matrixm.asm('sw',19,4*32,24)# store res[i, j] from x[19]m.asm('bne',21,0,'inner-loop')# branch to 'inner-loop' if x[21] != 0m.asm('bne',20,0,'outer-loop')# branch to 'outer-loop' if x[20] != 0m.lbl('end')# execute program from 'start' to 'end'm.exe(start='start',end='end')# compare results against golden referenceres=m.read_i32_vec(4*32,size=4*4).reshape(4,4)# read result matrixref=np.matmul(A,B)# golden referenceprint(np.array_equal(res,ref))# should return 'True'# Output: TrueExample 3.3:Same as example 3.2,  but now use Python for-loops in the assembly code to improve readability.# generate 4x4 matrices A and B and store them in memoryA=np.random.randint(100,size=(4,4))B=np.random.randint(100,size=(4,4))m.write_i32_vec(A.flatten(),0)# write matrix A to mem[0]m.write_i32_vec(B.flatten(),4*16)# write matrix B to mem[4*16]# store assembly program starting at address 4*128m.pc=4*128# here, we decrement the loop variables down to 0 so that we don't need an# additional register to hold the constant for detecting the end of the loop:#  - x[20] is 4*4*i (i.e. the outer-loop variable) and is decremented by 16 from 64#  - x[21] is 4*j (i.e. the inner-loop variable) and is decremented by 4 from 16m.lbl('start')m.asm('addi',20,0,64)# x[20] := 0 + 64m.lbl('outer-loop')m.asm('addi',20,20,-16)# decrement loop-variable: x[20] := x[20] - 16forkinrange(4):m.asm('lw',10+k,k*4,20)# load x[10+k] with A[i, k] from mem[k*4 + x[20]]m.asm('addi',21,0,16)# reset loop-variable j: x[21] := 0 + 16m.lbl('inner-loop')m.asm('addi',21,21,-4)# decrement j: x[21] := x[21] - 4m.asm('lw',18,4*16,21)# load x[18] with B[0, j] from mem[4*16 + x[21]]m.asm('mul',19,10,18)# x[19] := x[10] * x[18] = A[i, 0] * B[0, j]forkinrange(1,4):m.asm('lw',18,4*(16+k*4),21)# load x[18] with B[k, j]m.asm('mul',18,10+k,18)# x[18] := x[10+k] * x[18] = A[i, k] * B[k, j]m.asm('add',19,19,18)# x[19] := x[19] + x[18]m.asm('add',24,20,21)# calculate base address for result-matrixm.asm('sw',19,4*32,24)# store res[i, j] from x[19]m.asm('bne',21,0,'inner-loop')# branch to 'inner-loop' if x[21] != 0m.asm('bne',20,0,'outer-loop')# branch to 'outer-loop' if x[20] != 0m.lbl('end')# execute program from 'start' to 'end'm.exe(start='start',end='end')# compare results against golden referenceres=m.read_i32_vec(4*32,size=4*4).reshape(4,4)# read result matrixref=np.matmul(A,B)# golden referenceprint(np.array_equal(res,ref))# should return 'True'# Output: TruePerformance numbers for example 3.3:>>>m.print_perf()Opscounters:{'total':269,'load':80,'store':16,'mul':64,'add':89,'madd':0,'branch':20}x[]regfile:9outof31x-registersareusedf[]regfile:0outof32f-registersareusedImagesize:92BytesExample 3.4:4x4 matrix multiplication optimized for runtime at the expense of image size and register file usage. Specifically, we first store the entire B matrix in the register file. And we fully unroll the for-loops to eliminate loop variables and branch instructions at the expense of a larger image size.# generate 4x4 matrices A and B and store them in memoryA=np.random.randint(100,size=(4,4))B=np.random.randint(100,size=(4,4))m.write_i32_vec(A.flatten(),0)# write matrix A to mem[0]m.write_i32_vec(B.flatten(),4*16)# write matrix B to mem[4*16]# store assembly program starting at address 4*128m.pc=4*128m.lbl('start')# load entire B matrix into registers x[16] ... x[31]foriinrange(4):forjinrange(4):m.asm('lw',16+4*i+j,4*(16+4*i+j),0)# perform matmul in row-major orderforiinrange(4):forkinrange(4):# load x[10] ... x[13] with row i of Am.asm('lw',10+k,4*(4*i+k),0)# load x[10+k] with A[i, k]forjinrange(4):m.asm('mul',15,10,16+j)# x[15] := x[10] * x[16+j] = A[i, 0] * B[0, j]forkinrange(1,4):m.asm('mul',14,10+k,16+4*k+j)# x[14] := x[10+k] * x[16+4k+j] = A[i, k] * B[k, j]m.asm('add',15,15,14)# x[15] := x[15] + x[14]m.asm('sw',15,4*(32+i*4+j),0)# store res[i, j] from x[15]m.lbl('end')# execute program from 'start' to 'end'm.exe(start='start',end='end')# compare results against golden referenceres=m.read_i32_vec(4*32,size=4*4).reshape(4,4)# read result matrixref=np.matmul(A,B)# golden referenceprint(np.array_equal(res,ref))# should return 'True'# Output: TrueThe table below shows a speedup of 1.7 with the following caveats:The bit-widths don't make sense for fixed point (in general, multiplying two 32-bit integers produces a 64-bit product; and adding 4 of these products requires up to 66 bits).For runtime calculations, we assume that our RISC-V CPU can only perform one instruction per cycle (while many RISC-V cores can perform multiple instructions per cycle).We assume all 31 registers can be used, which is unrealistic because we ignore register allocation conventions such as the procedure
calling conventions specifiedhere.ImageRegistersLoadStoreMulAddBranchTotal opsSpeedupExample 3.392B980166489202691Example 3.4640B223216644801601.7Example 4: Neural network layersComing soon, seefile layer_examples.pyfor nowExample 5: MobileNetComing soon-ish, seefile mobilenet_v1_0.25.pyfor nowRunning in colabThis is the quickest way to get started and should work on any machine.If you have a free Google Drive account, you can make a copy of this colab via the menuFile->Save a copy in Drive. Now you can edit the code.Alternatively, start a new colab in your Google Drive as follows:Go hereand click onNew->More->Google Colaboratory. Then copy below lines into your colab:!pipinstalltinyfivefromtinyfive.machineimportmachineimportnumpyasnpm=machine(mem_size=4000)# instantiate RISC-V machine with 4KB of memoryRunning without packageIf you don't want to use the TinyFive python package, then you can clone the latest repo and install numpy as follows:gitclonehttps://github.com/OpenMachine-ai/tinyfive.gitcdtinyfive
pipinstallnumpyTo run the examples, type:python3examples.pyTo run the test suite, type:python3tests.pyIf you don't want to run above steps on your local machine, you can run it in a colab as follows: Start a new colab in your Google Drive bygoing hereand clicking onNew->More->Google Colaboratory. Then copy below lines into your colab:!gitclonehttps://github.com/OpenMachine-ai/tinyfive.git%cdtinyfive# run examples!python3examples.py# run test suite!python3tests.pyContributeIf you like this project, give it a ⭐ and share it with friends!  And if you are interested in helping make TinyFive better,
I highly welcome you to do so. I thank you in advance for your interest.  If you are unsure of what you could do to improve the project, you may have a lookhere.Latest statusTinyFive is still under construction, many things haven't been implemented and tested yet.37 of the 40 base instructions (RV32I), all instructions of the M-extension (RV32M) and the F-extension (RV32F) with the default rounding mode are already implemented, and many of them are tested.  (The three missing RV32I instructionsfence,ebreak, andecallare not applicable here.)Remaining work: improve testing, add more extensions. See TODOs in the code for more details.Stay updated by following us onTwitter,Post.news, andLinkedIn.SpeedTinyFive is not optimized for speed (but for ease-of-use andLOC).You might be able to use PyPy orCodonto speed up TinyFive (see e.g. thePydgin paperfor details).If you only use the upper-case instructions such asADD(), then TinyFive is very fast because there is no instruction decoding. And you should be able to accelerate it on a GPU or TPU.If you use the lower-case instructions withasm()andexe(), then execution of these functions is slow as they involve look-up and string matching with O(n) complexity where "n" is the total number of instructions. The current implementations ofasm()anddec()are optimized for ease-of-use and readability. A faster implementation would collapse multiple look-ups into one look-up, optimize the pattern-matching for the instruction decoding (bits -> instruction), and change the order of the instructions so that more frequently used instructions are at the top of the list.Here is an older versionof TinyFive with a fasterdec()function that collapses two look-ups (bits -> instructionandinstruction -> uppeer-case instruction) and doesn't usefnmatch.ComparisonThe table below compares TinyFive with otherISSand emulator projects.ISSAuthorLanguageMature?ExtensionsLOCTinyFiveOpenMachinePythonNoI, M, some F< 1kPydginCornell UniversityPython, CLast update 2016A, D, F, I, MSpikeUC BerkeleyC, C++YesAllQEMUFabrice BellardCYesAllTinyEMUFabrice BellardCYesAllriscvOVPsimImperasCYesAllWhisperWestern DigitalC, C++YesAlmost allSail ModelCambridge, EdinburghSail, CYesAllPiMaker/rvcPiMakerCmini-rv32imaCharles LohrCA, I, M, Zifencei, Zicsr< 1kReferencesHuggingFive:raised_hand_with_fingers_splayed:OfficialRISC-V specSeethis RISC-V cardfor a brief description of most instructions. See also theRISC-V reference card.BookThe RISC-V Reader: An Open Architecture Atlasby David Patterson and Andrew Waterman. Appendix A of this book defines all instructions. The Spanish version of this book isavailable for free,
other free versions areavailable here.PydginpaperandvideoOnline simulatorfor debugTiny Tech promiseSimilar toTinyEMU,tinygrad, and other “tiny tech” projects, we believe that core technology should be simple and small (in terms of LOC). Therefore, we will make sure that the core of TinyFive (without tests and examples) will always be below 1000 lines.Simplicity and size (in terms of number of instructions) is a key feature ofRISC: the "R" in RISC stands for "reduced" (as opposed to complex CISC). Specifically, the ISA manual of RISC-V has only ~200 pages while the ARM-32 manual is over 2000 pages long according to Fig. 1.6 of
theRISC-V Reader.