Package: upysize
Decreasing micropython code sizeHaving smaller code is always a good thing. Sometimes it is a necessity, for example when running on a micro-controller with limited flash space. Smaller code should also be quicker to execute, due to less instructions.This is a collection of tips and tricks to reduce the compiled size of themicropythoncode.Majority of these strategies are also automated into this tool, so one does not need to search for them manually.One of the very few resources on this topic ismicropython docs- even though it mainly concerns speed, some tips are common for code size efficiency.All the examples below are real, coming frommicropythoncodebase ofTrezor firmwarerepository.(Want to measure how big is the code in the final binary? Try outsize analysis tool)Tool basic usageInstalling this package createsupysizecommand, which takes the file/directory as an argument.$upysize.
$upysizesrc/apps/bitcoin/sign_tx/bitcoin.py
$upysizesrc/appsResult will be printed into terminal, suggesting some changes in specific files, together with estimated decrease of binary size.StrategiesOverall, strategies have one thing in common - reducing the amount of instructions/bytecode that need to be compiled into the final binary/executable.To sum up some of the lessons learned:Accessing global symbols from functions is expensive.
Accessing symbol attributes is expensive.which leads to a sayingCache is our friend.List below shows the strategies (in no particular order).1. Delete unused codeProbably the easiest way to reduce the size of the code is to delete all the unused stuff (imports, functions, classes, variables... ) - or at least delete it from the resulting binary. How to search for it? Apart from doing it manually, what helps are two things:code editor (at leastVS Code) highlights unused symbols - they become greyed outtools likeVulturecan analyze wholepythoncodebase and report all possibly unused symbols2. Import only what is needed for the code executionStatic type hints in modernpythonare a fantastic thing, but they are not without some drawbacks. When type-hinting with object from another module, this object needs to be imported - and therefore, normally, will generate some runtime and compiled overhead.To combat this (and also to combat circular dependencies), it is possible to usetyping.TYPE_CHECKINGflag. This flag isFalseat runtime, butTruewhen type-checking the code. The imports insideif TYPE_CHECKING:are not executed at runtime (not included into binary at compile time).Beforefrommy_moduleimportMyObjectdefmy_func(obj:MyObject)->None:...AfterfromtypingimportTYPE_CHECKINGifTYPE_CHECKING:frommy_moduleimportMyObjectdefmy_func(obj:MyObject)->None:...BenefitsSize cost of one global import is around8 bytes.3. Inline short one-time helper functionsHaving a lot of small and well-documented helper functions is generally a good practice. However, each function has significant size overhead in the resulting binary.When a function is used only once, it can be beneficial to inline it. This is especially true for short functions, which are not used anywhere else.Beforedefderive_shelley_address(parameters:Params)->bytes:header=_create_header(parameters.address_type,network_id)...def_create_header(address_type:AddressType,network_id:int)->bytes:header=address_type<<4|network_idreturnheader.to_bytes(1,"little")Afterdefderive_shelley_address(parameters:Params)->bytes:# _create_headerheader_int=parameters.address_type<<4|network_idheader=header_int.to_bytes(1,"little")...BenefitsSize benefit of inlining one-time function is around50 bytes. It depends on the amount of function arguments - the more, the bigger the size decrease. Also, those two functions can now share one local scope, which is beneficial for caching purposes (see next strategies).DrawbacksIt can decrease the readability of the code. It is also then impossible to test that helper function in isolation. What would solve both these issues is compile-time magic doing the "inlining" automatically.4. Create functions of repeating patternsIt may be a contradiction of the point above, to create more functions, but when the same logic is used many times, it can be (not only space-wise) beneficial to group it into a function.Beforeself.prevouts=HashWriter(blake2b(outlen=32,personal=b"ZTxIdPrevoutHash"))self.amounts=HashWriter(blake2b(outlen=32,personal=b"ZTxTrAmountsHash"))...self.outputs=HashWriter(blake2b(outlen=32,personal=b"ZTxIdOutputsHash"))Afterdefblake_hash_writer_32(personal:bytes)->HashWriter:returnHashWriter(blake2b(outlen=32,personal=personal))self.prevouts=blake_hash_writer_32(b"ZTxIdPrevoutHash")self.amounts=blake_hash_writer_32(b"ZTxTrAmountsHash")...self.outputs=blake_hash_writer_32(b"ZTxIdOutputsHash")BenefitsThe more expensive the operation is, the bigger the size benefit. Also it is more apparent that all the steps are really the same in all places.5. Replace series of calls with a loopWhen there is an uninterrupted sequence of the same calls, just with different arguments, it can be beneficial to replace it with a loop.Beforewrite_uint32_le(w,tx.version|OVERWINTERED)# nVersion | fOverwinteredwrite_uint32_le(w,tx.version_group_id)# nVersionGroupIdwrite_uint32_le(w,tx.branch_id)# nConsensusBranchIdwrite_uint32_le(w,tx.lock_time)# lock_timewrite_uint32_le(w,tx.expiry)# expiryHeightAfterfornumin(tx.version|_OVERWINTERED,# nVersion | fOverwinteredtx.version_group_id,# nVersionGroupIdtx.branch_id,# nConsensusBranchIdtx.lock_time,# lock_timetx.expiry,# expiryHeight):write_uint32_le(w,num)6. Not setting values through if-elif-elseIf applicable (when always comparing the same thing), adictmay be more efficient than a longif-elif-elsechain.Beforeifrotation==0:label="north"elifrotation==90:label="east"elifrotation==180:label="south"elifrotation==270:label="west"else:raisewire.DataError("Unsupported display rotation")Afterlabel={0:"north",90:"east",180:"south",270:"west",}.get(rotation)iflabelisNone:raiseDataError("Unsupported display rotation")In very simple cases, ternary operator can be even more space-efficient:ifmsg.fee.change>0:action="increase"else:action="decrease"vsaction="increase"ifmsg.fee.change>0else"decrease"7. Cache any frequent attribute accessAttribute access is expensive, so when we do it repeatedly, we could cache the result into one variable.Before(focus onmsg.address_n)asyncdefget_address(ctx:Context,msg:GetAddress)->Address:awaitvalidate(ctx,msg.address_n)node=derive(msg.address_n)address=to_str(msg.address_n)returnAddress(address=address)Afterasyncdefget_address(ctx:Context,msg:GetAddress)->Address:address_n=msg.address_n# cacheawaitvalidate(ctx,address_n)node=derive(address_n)address=to_str(address_n)returnAddress(address=address)It works well even with methods:my_list=[]my_list.append(...)...my_list.append(...)vsmy_list=[]my_list_append=my_list.append# cachemy_list_append(...)...my_list_append(...)BenefitsThe more the attribute lookup was used, the bigger the saving. It saves around2 bytesper one replaced lookup (when replacing 4 or more usages).DrawbacksFor caching methods, it worsens their readability in code editor (my_list.appendhas object and method differentiated by color, whilemy_list_appendis only a variable).8. Cache frequently accessed global symbol locallyAccessing global symbols (imported modules or top-level functions) is expensive. When we use it many times inside one function, it could be beneficial to store the reference in a local variable.Beforefromtrezor.messagesimportMessageTypedefboot()->None:register(MessageType.GetAddress,get_address)register(MessageType.GetPublicKey,get_public_key)...register(MessageType.Ping,ping)Afterfromtrezor.messagesimportMessageTypedefboot()->None:MT=MessageType# cacheregister(MT.GetAddress,get_address)register(MT.GetPublicKey,get_public_key)...register(MT.Ping,ping)"Double win" is storing a local reference to a global attribute lookup:fromtrezorimportwiredefvalidate(msg:Message)->None:ifmsg.high_fee:raisewire.DataError("high_fee")ifmsg.low_fee:raisewire.DataError("low_fee")...vsfromtrezorimportwiredefvalidate(msg:Message)->None:DataError=wire.DataError# cacheifmsg.high_fee:raiseDataError("high_fee")ifmsg.low_fee:raiseDataError("low_fee")...BenefitsThe more global usages are replaced, the bigger benefit. It is roughly1 byteper replacement when replacing 5 or more usages.9. Import symbols only in the one function where it is usedWhen some imported symbol is used only in one function from the module, it is beneficial to import it locally just in that single function instead of creating a global import.Beforefromtrezor.crypto.hashlibimportsha256defget_tx_hash(w:HashWriter)->bytes:d=w.get_digest()returnsha256(sha256(d).digest()).digest()Afterdefget_tx_hash(w:HashWriter)->bytes:fromtrezor.crypto.hashlibimportsha256d=w.get_digest()returnsha256(sha256(d).digest()).digest()BenefitsThe more times the symbol is used in that function, the more space-beneficial it is to import it just there (as it creates a local symbol, not a global one).  For each usage of local import, around2-3 bytesare saved.DrawbacksThis way not all the imports are on the top and it is therefore harder to see all the dependencies of the module. Also when later adding another function using this symbol, we need to decide whether to import it also locally there, or do it globally.10. Useconstfor global constantsCreating a constant is more costly than to define a normal number, but it saves a lot in places which use this constant.BeforeHASH_LENGTH=32SLIP_44_ID=123LOCAL_ONLY=456AfterfrommicropythonimportconstHASH_LENGTH=const(32)SLIP_44_ID=const(123)_LOCAL_ONLY=const(456)One extra benefit of usingconst()is when the variable is used only in module where it is defined. In that case, prepending a variable name with_will tellmicropythonthat it does not need to be included in the public module dictionary. This saves around4 bytes.  More details inmicropython docs.11. Import frequently used symbols directlyTo avoid the need of attribute lookups on imported modules, importing the used symbols directly can be beneficial when that symbol is used many times in more than one function.Beforefromtrezorimportwireimporttrezorui2defshow_group_share_success():...ifresultisnottrezorui2.CONFIRMED:raisewire.ActionCancelleddefcontinue_recovery():...ifresultisnottrezorui2.CONFIRMED:raisewire.ActionCancelledAfterfromtrezor.wireimportActionCancelledfromtrezorui2importCONFIRMED# CONFIRMED = trezorui2.CONFIRMED  # cache  # second optiondefshow_group_share_success():...ifresultisnotCONFIRMED:raiseActionCancelleddefcontinue_recovery():...ifresultisnotCONFIRMED:raiseActionCancelled12. Do not use keyword argumentsCalling functions with keyword arguments is nice (helloRust), but it turns out it costs more space than with positional arguments.Beforemosaic=MosaicLevy(type=Mosaic.ABC,fee=10,namespace="dim",mosaic="coin",)Aftermosaic=MosaicLevy(# levyMosaic.ABC,# type10,# fee"dim",# namespace"coin",# mosaic)BenefitsIt saves3 bytesper one kwarg pair.DrawbacksIt can decrease readability of what each argument means, especially when there are many of them. It can be therefore good to include comments with the names of the arguments.13. Do not use small classes to hold data for a short timeWhen data have only short-term duration (are not passed to other modules, etc.), the class may be replaced by a tuple holding just the raw data.Defining class and creating an instance are both quite costly, so avoiding it saves space. Tuples do not have this cost.BeforeclassRippleField:def__init__(self,type:int,key:int)->None:self.type:int=typeself.key:int=keyFIELDS:list[RippleField]=[RippleField(type=FIELD_TYPE_ACCOUNT,key=1),RippleField(type=FIELD_TYPE_ACCOUNT,key=3),...RippleField(type=FIELD_TYPE_INT32,key=27),]defserialize(w:Writer):forfieldinFIELDS:write(w,field)defwrite(w:Writer,field:RippleField)->None:type=field.typekey=field.key...AfterifTYPE_CHECKING:RippleField=tuple[int,int]FIELDS:list[RippleField]=[(FIELD_TYPE_ACCOUNT,1),(FIELD_TYPE_ACCOUNT,3),...(FIELD_TYPE_INT32,27),]defserialize(w:Writer):forfieldinFIELDS:write(w,field)defwrite(w:Writer,field:RippleField)->None:type=field[0]key=field[1]...DrawbacksIt worsens the readability/reliability due the need of using number subscripting (field[0]) to access the data. It can be therefore good to define a type alias for the tuple to have a static analysis.14. Effectively store bigger dataIn memory-constrained environments we sometimes need to work with data-sets that cannot even fit RAM in their entirety. For those cases the data need to be stored in flash and accessed sequentially, which can cost a lot of space.Yielding iterators of tuples turned out to be the most effective way. Some higher function will loop through them and can do something on each element.(Using tuples is more efficient than classes, as already discussed above. Class can be constructed by the higher function.)BeforeclassTokenInfo:def__init__(self,symbol:str,decimals:int)->None:self.symbol=symbolself.decimals=decimalsdeftoken_by_chain_address(address:bytes)->TokenInfo:ifaddress==b"\x4e\x84...":returnTokenInfo("$FFC",18)ifaddress==b"\x7d\xd7...":returnTokenInfo("$TEAK",18)...Afterdeftoken_by_chain_address(address:bytes)->TokenInfo:fortin_token_iterator():ifaddress==t[0]:returnTokenInfo(t[1],t[2])def_token_iterator()->Iterator[tuple[bytes,str,int]]:yield(# address, symbol, decimalsb"\x4e\x84...","$FFC",18,)yield(# address, symbol, decimalsb"\x7d\xd7..","$TEAK",18,)...ConclusionUsing the strategies above, it was possible to shrink the compiled size ofTrezor firmware'smicropythoncode by around10 %.