Answer: The hash table implementation of dictionaries uses a hash value calculated from the key value to find the key  If the key were a mutable object its value could change and thus its hash could also change  But since whoever changes the key object cant tell that it was being used as a dictionary key it cant move the entry around in the dictionary  Then when you try to look up the same object in the dictionary it wont be found because its hash value is different If you tried to look up the old value it wouldnt be found either because the value of the object found in that hash bin would be differentIf you want a dictionary indexed with a list simply convert the list to a tuple first the functiontupleLcreates a tuple with the same entries as the listL  Tuples are immutable and can therefore be used as dictionary keysSome unacceptable solutions that have been proposedHash lists by their address object ID  This doesnt work because if you construct a new list with the same value it wont be found egmydict1212printmydict12would raise aKeyErrorexception because the id of the12used in the second line differs from that in the first line  In other words dictionary keys should be compared using not usingisMake a copy when using a list as a key  This doesnt work because the list being a mutable object could contain a reference to itself and then the copying code would run into an infinite loopAllow lists as keys but tell the user not to modify them  This would allow a class of hardtotrack bugs in programs when you forgot or modified a list by accident It also invalidates an important invariant of dictionaries every value indkeysis usable as a key of the dictionaryMark lists as readonly once they are used as a dictionary key  The problem is that its not just the toplevel object that could change its value you could use a tuple containing a list as a key  Entering anything as a key into a dictionary would require marking all objects reachable from there as readonly  and again selfreferential objects could cause an infinite loopThere is a trick to get around this if you need to but use it at your own risk You can wrap a mutable structure inside a class instance which has both aeqand ahashmethod You must then make sure that the hash value for all such wrapper objects that reside in a dictionary or other hash based structure remain fixed while the object is in the dictionary or other structureclassListWrapperdefinitselfthelistselfthelistthelistdefeqselfotherreturnselfthelistotherthelistdefhashselflselfthelistresult98767lenl555forielinenumerateltryresultresulthashel99999991001iexceptExceptionresultresult7777777i333returnresultNote that the hash computation is complicated by the possibility that some members of the list may be unhashable and also by the possibility of arithmetic overflowFurthermore it must always be the case that ifo1o2ieo1eqo2isTrue thenhasho1hasho2ieo1hasho2hash regardless of whether the object is in a dictionary or not  If you fail to meet these restrictions dictionaries and other hash based structures will misbehaveIn the case ofListWrapper whenever the wrapper object is in a dictionary the wrapped list must not change to avoid anomalies  Dont do this unless you are prepared to think hard about the requirements and the consequences of not meeting them correctly  Consider yourself warned

Question: Why must dictionary keys be immutable?


Answer: Lists and tuples while similar in many respects are generally used in fundamentally different ways  Tuples can be thought of as being similar to Pascalrecordsor Cstructs theyre small collections of related data which may be of different types which are operated on as a group  For example a Cartesian coordinate is appropriately represented as a tuple of two or three numbersLists on the other hand are more like arrays in other languages  They tend to hold a varying number of objects all of which have the same type and which are operated on onebyone  For exampleoslistdirreturns a list of strings representing the files in the current directory  Functions which operate on this output would generally not break if you added another file or two to the directoryTuples are immutable meaning that once a tuple has been created you cant replace any of its elements with a new value  Lists are mutable meaning that you can always change a lists elements  Only immutable elements can be used as dictionary keys and hence only tuples and not lists can be used as keys

Question: Why are there separate tuple and list data types?


Answer: The idea was borrowed from Modula3  It turns out to be very useful for a variety of reasonsFirst its more obvious that you are using a method or instance attribute instead of a local variable  Readingselfxorselfmethmakes it absolutely clear that an instance variable or method is used even if you dont know the class definition by heart  In C you can sort of tell by the lack of a local variable declaration assuming globals are rare or easily recognizable  but in Python there are no local variable declarations so youd have to look up the class definition to be sure  Some C and Java coding standards call for instance attributes to have anmprefix so this explicitness is still useful in those languages tooSecond it means that no special syntax is necessary if you want to explicitly reference or call the method from a particular class  In C if you want to use a method from a base class which is overridden in a derived class you have to use theoperator  in Python you can writebaseclassmethodnameselfargumentlist  This is particularly useful forinitmethods and in general in cases where a derived class method wants to extend the base class method of the same name and thus has to call the base class method somehowFinally for instance variables it solves a syntactic problem with assignment since local variables in Python are by definition those variables to which a value is assigned in a function body and that arent explicitly declared global there has to be some way to tell the interpreter that an assignment was meant to assign to an instance variable instead of to a local variable and it should preferably be syntactic for efficiency reasons  C does this through declarations but Python doesnt have declarations and it would be a pity having to introduce them just for this purpose  Using the explicitselfvarsolves this nicely  Similarly for using instance variables having to writeselfvarmeans that references to unqualified names inside a method dont have to search the instances directories  To put it another way local variables and instance variables live in two different namespaces and you need to tell Python which namespace to use

Question: Why must self be used explicitly in method definitions and calls?


Answer: bjects referenced from the global namespaces of Python modules are not always deallocated when Python exits  This may happen if there are circular references  There are also certain bits of memory that are allocated by the C library that are impossible to free eg a tool like Purify will complain about these  Python is however aggressive about cleaning up memory on exit and does try to destroy every single objectIf you want to force Python to delete certain things on deallocation use theatexitmodule to run a function that will force those deletions

Question: Why isn t all memory freed when cpython exits?


Answer: re precisely they cant end with an odd number of backslashes the unpaired backslash at the end escapes the closing quote character leaving an unterminated stringRaw strings were designed to ease creating input for processors chiefly regular expression engines that want to do their own backslash escape processing Such processors consider an unmatched trailing backslash to be an error anyway so raw strings disallow that  In return they allow you to pass on the string quote character by escaping it with a backslash  These rules work well when rstrings are used for their intended purposeIf youre trying to build Windows pathnames note that all Windows system calls accept forward slashes toofopenmydirfiletxt works fineIf youre trying to build a pathname for a DOS command try eg one ofdirrthisismydosdirdirrthisismydosdir 1dirthisismydosdir

Question: Why can t raw strings r strings end with a backslash?


Answer: Cythoncompiles a modified version of Python with optional annotations into C extensionsNuitkais an upandcoming compiler of Python into C code aiming to support the full Python language

Question: Can python be compiled to machine code c or some other language?


Answer: uido saida For some operations prefix notation just reads better than postfix  prefix and infix operations have a long tradition in mathematics which likes notations where the visuals help the mathematician thinking about a problem Compare the easy with which we rewrite a formula like xab into xa  xb to the clumsiness of doing the same thing using a raw OO notationb When I read code that says lenx Iknowthat it is asking for the length of something This tells me two things the result is an integer and the argument is some kind of container To the contrary when I read xlen I have to already know that x is some kind of container implementing an interface or inheriting from a class that has a standard len Witness the confusion we occasionally have when a class that is not implementing a mapping has a get or keys method or something that isnt a file has a write methodhttpsmailpythonorgpipermailpython30002006November004643html

Question: Why does python use methods for some functionality e g list index but functions for other e g len list?


Answer: CPythons dictionaries are implemented as resizable hash tables  Compared to Btrees this gives better performance for lookup the most common operation by far under most circumstances and the implementation is simplerDictionaries work by computing a hash code for each key stored in the dictionary using thehashbuiltin function  The hash code varies widely depending on the key and a perprocess seed for examplePythoncould hash to539294296whilepython a string that differs by a single bit could hash to1142331976  The hash code is then used to calculate a location in an internal array where the value will be stored  Assuming that youre storing keys that all have different hash values this means that dictionaries take constant time O1 in BigO notation  to retrieve a key

Question: How are dictionaries implemented in cpython?


Answer: In the 1970s people realized that unrestricted goto could lead to messy spaghetti code that was hard to understand and revise In a highlevel language it is also unneeded as long as there are ways to branch in Python withifstatements andorand andifelseexpressions and loop withwhileandforstatements possibly containingcontinueandbreakOne can also use exceptions to provide a structured goto that works even across function calls  Many feel that exceptions can conveniently emulate all reasonable uses of thegoorgotoconstructs of C Fortran and other languages  For exampleclasslabelExceptionpass declare a labeltryifconditionraiselabel goto labelexceptlabel where to gotopassThis doesnt allow you to jump into the middle of a loop but thats usually considered an abuse ofgotoanyway  Use sparingly

Question: Why is there no goto?


Answer: An interface specification for a module as provided by languages such as C and Java describes the prototypes for the methods and functions of the module  Many feel that compiletime enforcement of interface specifications helps in the construction of large programsPython 26 adds anabcmodule that lets you define Abstract Base Classes ABCs  You can then useisinstanceandissubclassto check whether an instance or a class implements a particular ABC  Thecollectionsabcmodule defines a set of useful ABCs such asIterableContainer andMutableMappingFor Python many of the advantages of interface specifications can be obtained by an appropriate test discipline for componentsA good test suite for a module can both provide a regression test and serve as a module interface specification and a set of examples  Many Python modules can be run as a script to provide a simple self test  Even modules which use complex external interfaces can often be tested in isolation using trivial stub emulations of the external interface  Thedoctestandunittestmodules or thirdparty test frameworks can be used to construct exhaustive test suites that exercise every line of code in a moduleAn appropriate testing discipline can help build large complex applications in Python as well as having interface specifications would  In fact it can be better because an interface specification cannot test certain properties of a program  For example thelistappendmethod is expected to add new elements to the end of some internal list an interface specification cannot test that yourlistappendimplementation will actually do this correctly but its trivial to check this property in a test suiteWriting test suites is very helpful and you might want to design your code to make it easily tested One increasingly popular technique testdriven development calls for writing parts of the test suite first before you write any of the actual code  Of course Python allows you to be sloppy and not write test cases at all

Question: How do you specify and enforce an interface spec in python?


Answer: Use the standard library modulesmtplibHeres a very simple interactive mail sender that uses it  This method will work on any host that supports an SMTP listenerimportsyssmtplibfromaddrinputFrom toaddrsinputTo splitprintEnter message end with DmsgwhileTruelinesysstdinreadlineifnotlinebreakmsgline The actual mail sendserversmtplibSMTPlocalhostserversendmailfromaddrtoaddrsmsgserverquitA Unixonly alternative uses sendmail  The location of the sendmail program varies between systems sometimes it isusrlibsendmail sometimesusrsbinsendmail  The sendmail manual page will help you out  Heres some sample codeimportosSENDMAILusrsbinsendmail sendmail locationpospopenst iSENDMAILwpwriteTo receiverexamplecomnpwriteSubject testnpwriten blank line separating headers from bodypwriteSome textnpwritesome more textnstspcloseifsts0printSendmail exit statussts

Question: How do i send mail from a python script?


Answer: he most common problem is that the signal handler is declared with the wrong argument list  It is called ashandlersignumframeso it should be declared with two parametersdefhandlersignumframe

Question: Why don t my signal handlers work?


Answer: Checkthe Library Referenceto see if theres a relevant standard library module  Eventually youll learn whats in the standard library and will be able to skip this stepFor thirdparty packages search thePython Package Indexor tryGoogleor another web search engine  Searching for Python plus a keyword or two for your topic of interest will usually find something helpful

Question: How do i find a module or application to perform task x?


Answer: You need to do two things the script files mode must be executable and the first line must begin withfollowed by the path of the Python interpreterThe first is done by executingchmodxscriptfileor perhapschmod755scriptfileThe second can be done in a number of ways  The most straightforward way is to writeusrlocalbinpythonas the very first line of your file using the pathname for where the Python interpreter is installed on your platformIf you would like the script to be independent of where the Python interpreter lives you can use theenvprogram  Almost all Unix variants support the following assuming the Python interpreter is in a directory on the usersPATHusrbinenv pythonDontdo this for CGI scripts  ThePATHvariable for CGI scripts is often very minimal so you need to use the actual absolute pathname of the interpreterOccasionally a users environment is so full that theusrbinenvprogram fails or theres no env program at all  In that case you can try the following hack due to Alex Rezinsky binshexecpython01The minor disadvantage is that this defines the scripts doc string However you can fix that by addingdocWhatever

Question: How do i make a python script executable on unix?


Answer: Thepydocmodule can create HTML from the doc strings in your Python source code  An alternative for creating API documentation purely from docstrings isepydocSphinxcan also include docstring content

Question: How do i create documentation from doc strings?


Answer: For Win32 OSX Linux BSD Jython IronPythonhttpspypiorgprojectpyserialFor Unix see a Usenet post by Mitch Chapmanhttpsgroupsgooglecomgroupsselm34A04430CF9ohioeecom

Question: How do i access the serial rs232 port?


Answer: or Unix variants The standard Python source distribution comes with a curses module in theModulessubdirectory though its not compiled by default Note that this is not available in the Windows distribution  there is no curses module for WindowsThecursesmodule supports basic curses features as well as many additional functions from ncurses and SYSV curses such as colour alternative character set support pads and mouse support This means the module isnt compatible with operating systems that only have BSD curses but there dont seem to be any currently maintained OSes that fall into this category

Question: Is there a curses termcap package for python?


Answer: Theshutilmodule contains acopyfilefunction Note that on Windows NTFS volumes it does not copyalternate data streamsnorresource forkson macOS HFS volumes though both are now rarely used It also doesnt copy file permissions and metadata though usingshutilcopy2instead will preserve most though not all of it

Question: How do i copy a file?


Answer: The standard modulerandomimplements a random number generator  Usage is simpleimportrandomrandomrandomThis returns a random floating point number in the range 0 1There are also many other specialized generators in this module such asrandrangeabchooses an integer in the range a buniformabchooses a floating point number in the range a bnormalvariatemeansdevsamples the normal Gaussian distributionSome higherlevel functions operate on sequences directly such aschoiceSchooses a random element from a given sequenceshuffleLshuffles a list inplace ie permutes it randomlyTheres also aRandomclass you can instantiate to create independent multiple random number generators

Question: How do i generate random numbers in python?


Answer: global interpreter lockGIL is used internally to ensure that only one thread runs in the Python VM at a time  In general Python offers to switch among threads only between bytecode instructions how frequently it switches can be set viasyssetswitchinterval  Each bytecode instruction and therefore all the C implementation code reached from each instruction is therefore atomic from the point of view of a Python programIn theory this means an exact accounting requires an exact understanding of the PVM bytecode implementation  In practice it means that operations on shared variables of builtin data types ints lists dicts etc that look atomic really areFor example the following operations are all atomic L L1 L2 are lists D D1 D2 are dicts x y are objects i j are intsLappendxL1extendL2xLixLpopL1ijL2LsortxyxfieldyDxyD1updateD2DkeysThese arentii1LappendL1LiLjDxDx1Operations that replace other objects may invoke those other objectsdelmethod when their reference count reaches zero and that can affect things  This is especially true for the mass updates to dictionaries and lists  When in doubt use a mutex

Question: What kinds of global value mutation are thread safe?


Answer: thonfile objectsare a highlevel layer of abstraction on lowlevel C file descriptorsFor most file objects you create in Python via the builtinopenfunctionfclosemarks the Python file object as being closed from Pythons point of view and also arranges to close the underlying C file descriptor  This also happens automatically infs destructor whenfbecomes garbageBut stdin stdout and stderr are treated specially by Python because of the special status also given to them by C  Runningsysstdoutclosemarks the Pythonlevel file object as being closed but doesnotclose the associated C file descriptorTo close the underlying C file descriptor for one of these three you should first be sure thats what you really want to do eg you may confuse extension modules trying to do IO  If it is useoscloseosclosestdinfilenoosclosestdoutfilenoosclosestderrfilenoOr you can use the numeric constants 0 1 and 2 respectively

Question: Why doesn t closing sys stdout stdin stderr really close it?


Answer: heglobal interpreter lockGIL is often seen as a hindrance to Pythons deployment on highend multiprocessor server machines because a multithreaded Python program effectively only uses one CPU due to the insistence that almost all Python code can only run while the GIL is heldBack in the days of Python 15 Greg Stein actually implemented a comprehensive patch set the free threading patches that removed the GIL and replaced it with finegrained locking  Adam Olsen recently did a similar experiment in hispythonsafethreadproject  Unfortunately both experiments exhibited a sharp drop in singlethread performance at least 30 slower due to the amount of finegrained locking necessary to compensate for the removal of the GILThis doesnt mean that you cant make good use of Python on multiCPU machines You just have to be creative with dividing the work up between multipleprocessesrather than multiplethreads  TheProcessPoolExecutorclass in the newconcurrentfuturesmodule provides an easy way of doing so themultiprocessingmodule provides a lowerlevel API in case you want more control over dispatching of tasksJudicious use of C extensions will also help if you use a C extension to perform a timeconsuming task the extension can release the GIL while the thread of execution is in the C code and allow other threads to get some work done  Some standard library modules such aszlibandhashlibalready do thisIt has been suggested that the GIL should be a perinterpreterstate lock rather than truly global interpreters then wouldnt be able to share objects Unfortunately this isnt likely to happen either  It would be a tremendous amount of work because many object implementations currently have global state For example small integers and short strings are cached these caches would have to be moved to the interpreter state  Other object types have their own free list these free lists would have to be moved to the interpreter state And so onAnd I doubt that it can even be done in finite time because the same problem exists for 3rd party extensions  It is likely that 3rd party extensions are being written at a faster rate than you can convert them to store all their global state in the interpreter stateAnd finally once you have multiple interpreters not sharing any state what have you gained over running each interpreter in a separate process

Question: Can t we get rid of the global interpreter lock?


Answer: Python comes with two testing frameworks  Thedoctestmodule finds examples in the docstrings for a module and runs them comparing the output with the expected output given in the docstringTheunittestmodule is a fancier testing framework modelled on Java and Smalltalk testing frameworksTo make testing easier you should use good modular design in your program Your program should have almost all functionality encapsulated in either functions or class methods  and this sometimes has the surprising and delightful effect of making the program run faster because local variable accesses are faster than global accesses  Furthermore the program should avoid depending on mutating global variables since this makes testing much more difficult to doThe global main logic of your program may be as simple asifnamemainmainlogicat the bottom of the main module of your programOnce your program is organized as a tractable collection of function and class behaviours you should write test functions that exercise the behaviours  A test suite that automates a sequence of tests can be associated with each module This sounds like a lot of work but since Python is so terse and flexible its surprisingly easy  You can make coding much more pleasant and fun by writing your test functions in parallel with the production code since this makes it easy to find bugs and even design flaws earlierSupport modules that are not intended to be the main module of a program may include a selftest of the moduleifnamemainselftestEven programs that interact with complex external interfaces may be tested when the external interfaces are unavailable by using fake interfaces implemented in Python

Question: How do i test a python program or component?


Answer: The easiest way is to use theconcurrentfuturesmodule especially theThreadPoolExecutorclassOr if you want fine control over the dispatching algorithm you can write your own logic manually  Use thequeuemodule to create a queue containing a list of jobs  TheQueueclass maintains a list of objects and has aputobjmethod that adds items to the queue and agetmethod to return them  The class will take care of the locking necessary to ensure that each job is handed out exactly onceHeres a trivial exampleimportthreadingqueuetime The worker thread gets jobs off the queue  When the queue is empty it assumes there will be no more work and exits Realistically workers will run until terminateddefworkerprintRunning workertimesleep01whileTruetryargqgetblockFalseexceptqueueEmptyprintWorkerthreadingcurrentthreadend printqueue emptybreakelseprintWorkerthreadingcurrentthreadend printrunning with argumentargtimesleep05 Create queueqqueueQueue Start a pool of 5 workersforiinrange5tthreadingThreadtargetworkernameworkerii1tstart Begin adding work to the queueforiinrange50qputi Give threads time to runprintMain thread sleepingtimesleep5When run this will produce the following outputRunning worker Running worker Running worker Running worker Running worker Main thread sleeping Worker Threadworker 1 started 130283832797456 running with argument 0 Worker Threadworker 2 started 130283824404752 running with argument 1 Worker Threadworker 3 started 130283816012048 running with argument 2 Worker Threadworker 4 started 130283807619344 running with argument 3 Worker Threadworker 5 started 130283799226640 running with argument 4 Worker Threadworker 1 started 130283832797456 running with argument 5 Consult the modules documentation for more details theQueueclass provides a featureful interface

Question: How do i parcel out work among a bunch of worker threads?


Answer: For Unix variants there are several solutions  Its straightforward to do this using curses but curses is a fairly large module to learn

Question: How do i get a single keypress at a time?


Answer: Useosremovefilenameorosunlinkfilename for documentation see theosmodule  The two functions are identicalunlinkis simply the name of the Unix system call for this functionTo remove a directory useosrmdir useosmkdirto create oneosmakedirspathwill create any intermediate directories inpaththat dont existosremovedirspathwill remove intermediate directories as long as theyre empty if you want to delete an entire directory tree and its contents useshutilrmtreeTo rename a file useosrenameoldpathnewpathTo truncate a file open it usingfopenfilenamerb and useftruncateoffset offset defaults to the current seek position  Theres alsoosftruncatefdoffsetfor files opened withosopen wherefdis the file descriptor a small integerTheshutilmodule also contains a number of functions to work on files includingcopyfilecopytree andrmtree

Question: How do i delete a file and other file questions?


Answer: See the chapters titledInternet Protocols and SupportandInternet Data Handlingin the Library Reference Manual  Python has many modules that will help you build serverside and clientside web systemsA summary of available frameworks is maintained by Paul Boddie athttpswikipythonorgmoinWebProgrammingCameron Laird maintains a useful set of pages about Python web technologies athttpswebarchiveorgweb20210224183619httpphaseitnetclairdcomplangpythonwebpython

Question: What www tools are there for python?


Answer: As soon as the main thread exits all threads are killed  Your main thread is running too quickly giving the threads no time to do any workA simple fix is to add a sleep to the end of the program thats long enough for all the threads to finishimportthreadingtimedefthreadtasknamenforiinrangenprintnameiforiinrange10TthreadingThreadtargetthreadtaskargsstriiTstarttimesleep10 But now on many platforms the threads dont run in parallel but appear to run sequentially one at a time  The reason is that the OS thread scheduler doesnt start a new thread until the previous thread is blockedA simple fix is to add a tiny sleep to the start of the run functiondefthreadtasknamentimesleep0001 foriinrangenprintnameiforiinrange10TthreadingThreadtargetthreadtaskargsstriiTstarttimesleep10Instead of trying to guess a good delay value fortimesleep its better to use some kind of semaphore mechanism  One idea is to use thequeuemodule to create a queue object let each thread append a token to the queue when it finishes and let the main thread read as many tokens from the queue as there are threads

Question: None of my threads seem to run why?


Answer: heatexitmodule provides a register function that is similar to Csonexit

Question: Is there an equivalent to c s onexit in python?


Answer: Theselectmodule is commonly used to help with asynchronous IO on socketsTo prevent the TCP connect from blocking you can set the socket to nonblocking mode  Then when you do theconnect you will either connect immediately unlikely or get an exception that contains the error number aserrnoerrnoEINPROGRESSindicates that the connection is in progress but hasnt finished yet  Different OSes will return different values so youre going to have to check whats returned on your systemYou can use theconnectexmethod to avoid creating an exception It will just return the errno value To poll you can callconnectexagain later 0orerrnoEISCONNindicate that youre connected  or you can pass this socket toselectselectto check if its writableNoteTheasynciomodule provides a general purpose singlethreaded and concurrent asynchronous library which can be used for writing nonblocking network code The thirdpartyTwistedlibrary is a popular and featurerich alternative

Question: How do i avoid blocking in the connect method of a socket?


Answer: Thepicklelibrary module solves this in a very general way though you still cant store things like open files sockets or windows and theshelvelibrary module uses pickle and gdbm to create persistent mappings containing arbitrary Python objects

Question: How do you implement persistent objects in python?


Answer: you cant find a source file for a module it may be a builtin or dynamically loaded module implemented in C C or other compiled language In this case you may not have the source file or it may be something likemathmodulec somewhere in a C source directory not on the Python PathThere are at least three kinds of modules in Pythonmodules written in Python pymodules written in C and dynamically loaded dll pyd so sl etcmodules written in C and linked with the interpreter to get a list of these typeimportsysprintsysbuiltinmodulenames

Question: Where is the math py socket py regex py etc source file?


Answer: You can find a collection of useful links on theWeb Programming wiki page

Question: What module should i use to help with generating html?


Answer: Be sure to use thethreadingmodule and not thethreadmodule Thethreadingmodule builds convenient abstractions on top of the lowlevel primitives provided by thethreadmodule

Question: How do i program using threads?


Answer: would like to retrieve web pages that are the result of POSTing a form Is there existing code that would let me do this easilyYes Heres a simple example that usesurllibrequestusrlocalbinpythonimporturllibrequest build the query stringqsFirstJosephineMIQLastPublic connect and send the server a pathrequrllibrequesturlopenhttpwwwsomeserverouttherecgibinsomecgiscriptdataqswithreqmsghdrsreqreadreqinfoNote that in general for percentencoded POST operations query strings must be quoted usingurllibparseurlencode  For example to sendnameGuySteeleJrimporturllibparseurllibparseurlencodenameGuy Steele JrnameGuySteele2CJrSee alsoHOWTO Fetch Internet Resources Using The urllib Packagefor extensive examples

Question: How can i mimic cgi form submission method post?


Answer: YesInterfaces to diskbased hashes such asDBMandGDBMare also included with standard Python  There is also thesqlite3module which provides a lightweight diskbased relational databaseSupport for most relational databases is available  See theDatabaseProgramming wiki pagefor details

Question: Are there any interfaces to database packages in python?


Answer: eadis a lowlevel function which takes a file descriptor a small integer representing the opened fileospopencreates a highlevel file object the same type returned by the builtinopenfunction Thus to readnbytes from a pipepcreated withospopen you need to usepreadn

Question: I can t seem to use os read on a pipe created with os popen why?


Answer: To read or write complex binary data formats its best to use thestructmodule  It allows you to take a string containing binary data usually numbers and convert it to Python objects and vice versaFor example the following code reads two 2byte integers and one 4byte integer in bigendian format from a fileimportstructwithopenfilenamerbasfsfread8xyzstructunpackhhlsThe  in the format string forces bigendian data the letter h reads one short integer 2 bytes and l reads one long integer 4 bytes from the stringFor data that is more regular eg a homogeneous list of ints or floats you can also use thearraymoduleNoteTo read and write binary data it is mandatory to open the file in binary mode here passingrbtoopen  If you userinstead the default the file will be open in text mode andfreadwill returnstrobjects rather thanbytesobjects

Question: How do i read or write binary data?


Answer: You can get a pointer to the module object as followsmodulePyImportImportModulemodulenameIf the module hasnt been imported yet ie it is not yet present insysmodules this initializes the module otherwise it simply returns the value ofsysmodulesmodulename  Note that it doesnt enter the module into any namespace  it only ensures it has been initialized and is stored insysmodulesYou can then access the modules attributes ie any name defined in the module as followsattrPyObjectGetAttrStringmoduleattrnameCallingPyObjectSetAttrStringto assign to variables in the module also works

Question: How do i access a module written in python from c?


Answer: Setup must end in a newline if there is no newline there the build process fails  Fixing this requires some ugly shell script hackery and this bug is so minor that it doesnt seem worth the effort

Question: I added a module using the setup file and the make fails why?


Answer: Sometimes you want to emulate the Python interactive interpreters behavior where it gives you a continuation prompt when the input is incomplete eg you typed the start of an if statement or you didnt close your parentheses or triple string quotes but it gives you a syntax error message immediately when the input is invalidIn Python you can use thecodeopmodule which approximates the parsers behavior sufficiently  IDLE uses this for exampleThe easiest way to do it in C is to callPyRunInteractiveLoopperhaps in a separate thread and let the Python interpreter handle the input for you You can also set thePyOSReadlineFunctionPointerto point at your custom input function SeeModulesreadlinecandParsermyreadlinecfor more hints

Question: How do i tell incomplete input from invalid input?


Answer: Call the functionPyRunStringfrom the previous question with the start symbolPyevalinput it parses an expression evaluates it and returns its value

Question: How can i evaluate an arbitrary python expression from c?


Answer: Python code define an object that supports thewritemethod  Assign this object tosysstdoutandsysstderr  Call printerror or just allow the standard traceback mechanism to work Then the output will go wherever yourwritemethod sends itThe easiest way to do this is to use theioStringIOclassimportiosyssysstdoutioStringIOprintfooprinthello worldsysstderrwritesysstdoutgetvaluefoohello worldA custom object to do the same would look like thisimportiosysclassStdoutCatcherioTextIOBasedefinitselfselfdatadefwriteselfstuffselfdataappendstuffimportsyssysstdoutStdoutCatcherprintfooprinthello worldsysstderrwritejoinsysstdoutdatafoohello world

Question: How do i catch the output from pyerr print or anything that prints to stdout stderr?


Answer: There are a number of alternatives to writing your own C extensions depending on what youre trying to doCythonand its relativePyrexare compilers that accept a slightly modified form of Python and generate the corresponding C code  Cython and Pyrex make it possible to write an extension without having to learn Pythons C APIIf you need to interface to some C or C library for which no Python extension currently exists you can try wrapping the librarys data types and functions with a tool such asSWIGSIPCXXBoost orWeaveare also alternatives for wrapping C libraries

Question: Writing c is hard are there any alternatives?


Answer: That depends on the objects type  If its a tuplePyTupleSizereturns its length andPyTupleGetItemreturns the item at a specified index  Lists have similar functionsPyListSizeandPyListGetItemFor bytesPyBytesSizereturns its length andPyBytesAsStringAndSizeprovides a pointer to its value and its length  Note that Python bytes objects may contain null bytes so Csstrlenshould not be usedTo test the type of an object first make sure it isntNULL and then usePyBytesCheckPyTupleCheckPyListCheck etcThere is also a highlevel API to Python objects which is provided by the socalled abstract interface  readIncludeabstracthfor further details  It allows interfacing with any kind of Python sequence using calls likePySequenceLengthPySequenceGetItem etc as well as many other useful protocols such as numbers PyNumberIndexet al and mappings in the PyMapping APIs

Question: How do i extract c values from a python object?


Answer: dynamically load g extension modules you must recompile Python relink it using g change LINKCC in the Python Modules Makefile and link your extension module using g eggsharedomymodulesomymoduleo

Question: How do i find undefined g symbols builtin new or pure virtual?


Answer: Yes you can create builtin modules containing functions variables exceptions and even new types in C  This is explained in the documentExtending and Embedding the Python InterpreterMost intermediate or advanced Python books will also cover this topic

Question: Can i create my own functions in c?


Answer: ou cant  UsePyTuplePackinstead

Question: How do i use py buildvalue to create a tuple of arbitrary length?


Answer: Most packaged versions of Python dont include theusrlibpython2xconfigdirectory which contains various files required for compiling Python extensionsFor Red Hat install the pythondevel RPM to get the necessary filesFor Debian runaptgetinstallpythondev

Question: I want to compile a python module on my linux system but some files are missing why?


Answer: The highestlevel function to do this isPyRunSimpleStringwhich takes a single string argument to be executed in the context of the modulemainand returns0for success and1when an exception occurred includingSyntaxError  If you want more control usePyRunString see the source forPyRunSimpleStringinPythonpythonrunc

Question: How can i execute arbitrary python statements from c?


Answer: es you can inherit from builtin classes such asintlistdict etcThe Boost Python Library BPLhttpswwwboostorglibspythondocindexhtml provides a way of doing this from C ie you can inherit from an extension class written in C using the BPL

Question: Can i create an object class with some methods implemented in c and others in python e g through inheritance?


Answer: I create my own functions in CYes using the C compatibility features found in C  PlaceexternCaround the Python include files and putexternCbefore each function that is going to be called by the Python interpreter  Global or static C objects with constructors are probably not a good idea


Answer: Depending on your requirements there are many approaches  To do this manually begin by readingthe Extending and Embedding document  Realize that for the Python runtime system there isnt a whole lot of difference between C and C  so the strategy of building a new Python type around a C structure pointer type will also work for C objectsFor C libraries seeWriting C is hard are there any alternatives

Question: How do i interface to c objects from python?


Answer: hePyObjectCallMethodfunction can be used to call an arbitrary method of an object  The parameters are the object the name of the method to call a format string like that used withPyBuildValue and the argument valuesPyObjectPyObjectCallMethodPyObjectobjectconstcharmethodnameconstcharargformatThis works for any object that has methods  whether builtin or userdefined You are responsible for eventuallyPyDECREFing the return valueTo call eg a file objects seek method with arguments 10 0 assuming the file object pointer is fresPyObjectCallMethodfseekii100ifresNULLanexceptionoccurredelsePyDECREFresNote that sincePyObjectCallObjectalwayswants a tuple for the argument list to call a function without arguments pass  for the format and to call a function with one argument surround the argument in parentheses eg i

Question: How do i call an object s method from c?


Answer: When using GDB with dynamically loaded extensions you cant set a breakpoint in your extension until your extension is loadedIn yourgdbinitfile or interactively add the commandbr PyImportLoadDynamicModuleThen when you run GDBgdblocalbinpythongdb run myscriptpygdb continue  repeat until your extension is loadedgdb finish    so that your extension is loadedgdb br myfunctionc50gdb continue

Question: How do i debug an extension?


Answer: Usually Python starts very quickly on Windows but occasionally there are bug reports that Python suddenly begins to take a long time to start up  This is made even more puzzling because Python will work fine on other Windows systems which appear to be configured identicallyThe problem may be caused by a misconfiguration of virus checking software on the problem machine  Some virus scanners have been known to introduce startup overhead of two orders of magnitude when the scanner is configured to monitor all reads from the filesystem  Try checking the configuration of virus scanning software on your systems to ensure that they are indeed configured identically McAfee when configured to scan all file system read activity is a particular offender

Question: Why does python sometimes take so long to start?


Answer: Embedding the Python interpreter in a Windows app can be summarized as followsDonotbuild Python into your exe file directly  On Windows Python must be a DLL to handle importing modules that are themselves DLLs  This is the first key undocumented fact  Instead link topythonNNdll it is typically installed inCWindowsSystemNNis the Python version a number such as 33 for Python 33You can link to Python in two different ways  Loadtime linking means linking againstpythonNNlib while runtime linking means linking againstpythonNNdll  General notepythonNNlibis the socalled import lib corresponding topythonNNdll  It merely defines symbols for the linkerRuntime linking greatly simplifies link options everything happens at run time  Your code must loadpythonNNdllusing the WindowsLoadLibraryExroutine  The code must also use access routines and data inpythonNNdllthat is Pythons C APIs using pointers obtained by the WindowsGetProcAddressroutine  Macros can make using these pointers transparent to any C code that calls routines in Pythons C APIIf you use SWIG it is easy to create a Python extension module that will make the apps data and methods available to Python  SWIG will handle just about all the grungy details for you  The result is C code that you linkintoyour exe file   You donothave to create a DLL file and this also simplifies linkingSWIG will create an init function a C function whose name depends on the name of the extension module  For example if the name of the module is leo the init function will be called initleo  If you use SWIG shadow classes as you should the init function will be called initleoc  This initializes a mostly hidden helper class used by the shadow classThe reason you can link the C code in step 2 into your exe file is that calling the initialization function is equivalent to importing the module into Python This is the second key undocumented factIn short you can use the following code to initialize the Python interpreter with your extension moduleincludePythonhPyInitialize Initialize PythoninitmyAppc Initialize import the helper classPyRunSimpleStringimport myApp Import the shadow classThere are two problems with Pythons C API which will become apparent if you use a compiler other than MSVC the compiler used to build pythonNNdllProblem 1 The socalled Very High Level functions that takeFILEarguments will not work in a multicompiler environment because each compilers notion of astructFILEwill be different  From an implementation standpoint these are very low level functionsProblem 2 SWIG generates the following code when generating wrappers to void functionsPyINCREFPyNoneresultobjPyNonereturnresultobjAlas PyNone is a macro that expands to a reference to a complex data structure called PyNoneStruct inside pythonNNdll  Again this code will fail in a multcompiler environment  Replace such code byreturnPyBuildValueIt may be possible to use SWIGstypemapcommand to make the change automatically though I have not been able to get this to work Im a complete SWIG newbieUsing a Python shell script to put up a Python interpreter window from inside your Windows app is not a good idea the resulting window will be independent of your apps windowing system  Rather you or the wxPythonWindow class should create a native interpreter window  It is easy to connect that window to the Python interpreter  You can redirect Pythons io to any object that supports read and write so all you need is a Python object defined in your extension module that contains read and write methods

Question: How can i embed python into a windows application?


Answer: This is not necessarily a straightforward question If you are already familiar with running programs from the Windows command line then everything will seem obvious otherwise you might need a little more guidanceUnless you use some sort of integrated development environment you will end uptypingWindows commands into what is referred to as a Command prompt window  Usually you can create such a window from your search bar by searching forcmd  You should be able to recognize when you have started such a window because you will see a Windows command prompt which usually looks like thisCThe letter may be different and there might be other things after it so you might just as easily see something likeDYourNameProjectsPythondepending on how your computer has been set up and what else you have recently done with it  Once you have started such a window you are well on the way to running Python programsYou need to realize that your Python scripts have to be processed by another program called the Pythoninterpreter  The interpreter reads your script compiles it into bytecodes and then executes the bytecodes to run your program So how do you arrange for the interpreter to handle your PythonFirst you need to make sure that your command window recognises the word py as an instruction to start the interpreter  If you have opened a command window you should try entering the commandpyand hitting returnCUsersYourNamepyYou should then see something likePython 364 v364d48eceb Dec 19 2017 060445 MSC v1900 32 bit Intel on win32Type help copyright credits or license for more informationYou have started the interpreter in interactive mode That means you can enter Python statements or expressions interactively and have them executed or evaluated while you wait  This is one of Pythons strongest features  Check it by entering a few expressions of your choice and seeing the resultsprintHelloHelloHello3HelloHelloHelloMany people use the interactive mode as a convenient yet highly programmable calculator  When you want to end your interactive Python session call theexitfunction or hold theCtrlkey down while you enter aZ then hit the Enter key to get back to your Windows command promptYou may also find that you have a Startmenu entry such asStart  Programs  Python 3x  Python command linethat results in you seeing theprompt in a new window  If so the window will disappear after you call theexitfunction or enter theCtrlZcharacter Windows is running a single python command in the window and closes it when you terminate the interpreterNow that we know thepycommand is recognized you can give your Python script to it Youll have to give either an absolute or a relative path to the Python script Lets say your Python script is located in your desktop and is namedhellopy and your command prompt is nicely opened in your home directory so youre seeing something similar toCUsersYourNameSo now youll ask thepycommand to give your script to Python by typingpyfollowed by your script pathCUsersYourName py Desktophellopy hello

Question: How do i run a python program under windows?


Answer: SeeHow can I create a standalone binary from a Python scriptfor a list of tools that can be used to make executables

Question: How do i make an executable from a python script?


Answer: Use themsvcrtmodule  This is a standard Windowsspecific extension module It defines a functionkbhitwhich checks whether a keyboard hit is present andgetchwhich gets one character without echoing it

Question: How do i check for a keypress without blocking?


Answer: occur on Python 35 and later when using Windows 81 or earlier without all updates having been installed First ensure your operating system is supported and is up to date and if that does not resolve the issue visit theMicrosoft support pagefor guidance on manually installing the C Runtime update

Question: How do i solve the missing api ms win crt runtime l1 1 0 dll error?


Answer: On Windows the standard Python installer already associates the py extension with a file type PythonFile and gives that file type an open command that runs the interpreter DProgramFilesPythonpythonexe1  This is enough to make scripts executable from the command prompt as foopy  If youd rather be able to execute the script by simple typing foo with no extension you need to add py to the PATHEXT environment variable

Question: How do i make python scripts executable?


Answer: The FAQ does not recommend using tabs and the Python style guidePEP 8 recommends 4 spaces for distributed Python code this is also the Emacs pythonmode defaultUnder any editor mixing tabs and spaces is a bad idea  MSVC is no different in this respect and is easily configured to use spaces TakeTools  Options  Tabs and for file type Default set Tab size and Indent size to 4 and select the Insert spaces radio buttonPython raisesIndentationErrororTabErrorif mixed tabs and spaces are causing problems in leading whitespace You may also run thetabnannymodule to check a directory tree in batch mode

Question: How do i keep editors from inserting tabs into my python source?


Question: Windows faq?


Answer: s pyd files are dlls but there are a few differences  If you have a DLL namedfoopyd then it must have a functionPyInitfoo  You can then write Python import foo and Python will search for foopyd as well as foopy foopyc and if it finds it will attempt to callPyInitfooto initialize it  You do not link your exe with foolib as that would cause Windows to require the DLL to be presentNote that the search path for foopyd is PYTHONPATH not the same as the path that Windows uses to search for foodll  Also foopyd need not be present to run your program whereas if you linked your program with a dll the dll is required  Of course foopyd is required if you want to sayimportfoo  In a DLL linkage is declared in the source code withdeclspecdllexport In a pyd linkage is defined in a list of available functions

Question: Is a pyd file the same as a dll?