Input:
How to write to an existing excel file without overwriting data (using pandas)? I use pandas to write to excel file in the following fashion:
import pandas

writer = pandas.ExcelWriter('Masterfile.xlsx')

data_filtered.to_excel(writer, "Main", cols=['Diff1', 'Diff2'])

writer.save()

Masterfile.xlsx already consists of number of different tabs.
Pandas correctly writes to "Main" sheet, unfortunately it also deletes all other tabs.

Output:
Pandas docs says it uses openpyxl for xlsx files. Quick look through the code in ExcelWriter gives a clue that something like this might work out:
import pandas
from openpyxl import load_workbook

book = load_workbook('Masterfile.xlsx')
writer = pandas.ExcelWriter('Masterfile.xlsx', engine='openpyxl')
writer.book = book
writer.sheets = dict((ws.title, ws) for ws in book.worksheets)

data_filtered.to_excel(writer, "Main", cols=['Diff1', 'Diff2'])

writer.save()


Input:
lxml runtime error: Reason: Incompatible library version: etree.so requires version 12.0.0 or later, but libxml2.2.dylib provides version 10.0.0 I have a perplexing problem.
I have used mac version 10.9, anaconda 3.4.1, python 2.7.6.
Developing web application with python-amazon-product-api. i have overcome an obstacle about installing lxml, referencing clang error: unknown argument: '-mno-fused-madd' (python package installation failure).
but another runtime error happened.
Here is the output from webbrowser.
Exception Type: ImportError
Exception Value:
dlopen(/Users/User_Name/Documents/App_Name/lib/python2.7/site-packages/lxml/etree.so, 2): Library not loaded: libxml2.2.dylib
Referenced from: /Users/User_Name/Documents/App_Name/lib/python2.7/site-packages/lxml/etree.so
Reason: Incompatible library version: etree.so requires version 12.0.0 or later, but libxml2.2.dylib provides version 10.0.0

Not sure how to proceed and have searched here and elsewhere for this particular error. Any help is much appreciated!

Output:
This worked for me:
brew install libxml2
brew install libxslt
brew link libxml2 --force
brew link libxslt --force


Input:
Are these two python statements the same? I have these two statements
return self.getData() if self.getData() else ''

and
return self.getData() or ''

I want to know are they same or there is any difference

Output: I would say No because if self.getData() changes something during its operation, then the first statement has the possibility of returning a different result since it will make a 2nd call to it.


Input:
How to convert XML to objects? I need to load an XML file and put the contents into an object-oriented structure. I want to take this:
<main>
    <object1 attr="name">content</object>
</main>

and turn it into something like this:
main
main.object1 = "content"
main.object1.attr = "name"

It'll have a more complicated structure than that and I can't hard code the element names. The names need to be collected when parsing and used as the object properties.
Any thoughts on what would be the best way to go about this?

Output:
It's worth to have a look at http://lxml.de/objectify.html
>>> xml = """<main>
... <object1 attr="name">content</object1>
... <object1 attr="foo">contenbar</object1>
... <test>me</test>
... </main>"""

>>> from lxml import objectify

>>> main = objectify.fromstring(xml)

>>> main.object1[0]
'content'

>>> main.object1[1]
'contenbar'

>>> main.object1[0].get("attr")
'name'

>>> main.test
'me'

Or the other way around to build xml structures:
>>> item = objectify.Element("item")

>>> item.title = "Best of python"

>>> item.price = 17.98

>>> item.price.set("currency", "EUR")

>>> order = objectify.Element("order")

>>> order.append(item)

>>> order.item.quantity = 3

>>> order.price = sum(item.price * item.quantity
... for item in order.item)

>>> import lxml.etree

>>> print lxml.etree.tostring(order, pretty_print=True)
<order>
  <item>
    <title>Best of python</title>
    <price currency="EUR">17.98</price>
    <quantity>3</quantity>
  </item>
  <price>53.94</price>
</order>


Input: Do dicts preserve iteration order if they are not modified? If I have a dictionary in Python, and I iterate through it once, and then again later, is the iteration order guaranteed to be preserved given that I didn't insert, delete, or update any items in the dictionary? (But I might have done look-ups).

Output:
Here is what dict.items() documentation says:

dict.items() return a copy of the dictionaryâs list of (key, value) pairs.
If items(), keys(), values(), iteritems(), iterkeys(), and itervalues() are called with no intervening modifications to the dictionary, the lists will directly correspond.

I think it's reasonable to assume that item ordering won't change if all you do is iteration.


Input:
How to catch an exception in python and get a reference to the exception, WITHOUT knowing the type? I'm wondering how I can catch any raised object (i.e. a type that does not extend Exception), and still get a reference to it.
I came across the desire to do this when using Jython.   When calling a Java method, if that method raises an exception, it will not extend Python's Exception class, so a block like this will not catch it:
try:
    # some call to a java lib that raises an exception here
except Exception, e:
    # will never be entered

I can do this, but then I have no access to the exception object that was raised.
try:
    # some call to a java lib that raises an exception here
except:
    # will enter here, but there's no reference to the exception that was raised

I can solve this by importing the Java exception type and catching it explicitly, but this makes it difficult/impossible to write generic exception handling wrappers/decorators.
Is there a way to catch some arbitrary exception and still get a reference to it in the except block?
I should note that I'm hoping for the exception handling decorator I am making to be usable with Python projects, not just with Jython projects.  I'd like to avoid importing java.lang.Exception because that just makes it Jython-only.  For example, I figure I can do something like this (but I haven't tried it), but I'd like to avoid it if I can.
try:
    # some function that may be running jython and may raise a java exception
except (Exception, java.lang.Exception), e:
    # I imagine this would work, but it makes the code jython-only

Output:
You can reference exceptions using the sys module. sys.exc_info is a tuple of the type, the instance and the traceback.
import sys

try:
    # some call to a java lib that raises an exception here
except:
    instance = sys.exc_info()[1]


Input:
Efficiently generate a 16-character, alphanumeric string I'm looking for a very quick way to generate an alphanumeric unique id for a primary key in a table.
Would something like this work?
def genKey():
    hash = hashlib.md5(RANDOM_NUMBER).digest().encode("base64")
    alnum_hash = re.sub(r'[^a-zA-Z0-9]', "", hash)
    return alnum_hash[:16]

What would be a good way to generate random numbers?
If I base it on microtime, I have to account for the possibility of several calls of genKey() at the same time from different instances.
Or is there a better way to do all this?
Thanks.

Output:
You can use this:
>>> import random
>>> ''.join(random.choice('0123456789ABCDEF') for i in range(16))
'E2C6B2E19E4A7777'

There is no guarantee that the keys generated will be unique so you should be ready to retry with a new key in the case the original insert fails. Also, you might want to consider using a deterministic algorithm to generate a string from an auto-incremented id instead of using random values, as this will guarantee you uniqueness (but it will also give predictable keys).


Input:
How to replace unicode characters by ascii characters in Python (perl script given)? I am trying to learn python and couldn't figure out how to translate the following perl script to python:
#!/usr/bin/perl -w

use open qw(:std :utf8);

while(<>) {
  s/\x{00E4}/ae/;
  s/\x{00F6}/oe/;
  s/\x{00FC}/ue/;
  print;
}

The script just changes unicode umlauts to alternative ascii output. (So the complete output is in ascii.) I would be grateful for any hints. Thanks!

Output:
For converting to ASCII you might want to try ASCII, Dammit or this recipe, which boils down to:
>>> title = u"KlÃ¼ft skrÃ¤ms infÃ¶r pÃ¥ fÃ©dÃ©ral Ã©lectoral groÃe"
>>> import unicodedata
>>> unicodedata.normalize('NFKD', title).encode('ascii','ignore')
'Kluft skrams infor pa federal electoral groe'


Input:
How to call a static methods on a django model class during a south migration I'm writing a data migration in south to fix some denormalized data I screwed up in earlier code.  The way to figure out the right value for the incorrect field is to call a static method on the django model class.  The code looks like this:
class Account(models.Model):
    name = models.CharField()

    @staticmethod
    def lookup_by_name(name):
        # There's actually more to it than this
        return Account.objects.get(name=name)

class Record(models.Model):
    account_name = models.CharField()
    acct = models.ForeignKey('Account')

...
class Migration(DataMigration):

    def forwards(self, orm):
        # Fixing Records with the wrong FK to Account
        for record in orm.Record.objects.all():
            record.acct = orm.Account.lookup_by_name(record.account_name)
            record.save()

But this fails with
AttributeError: type object 'Account' has no attribute 'lookup_by_name'
I'm guessing south just doesn't support @staticmethods on model classes?
Trying to import Account directly fails, unless I also import Record directly and completely ignore the ORM object.  Is that a safe option, since it's a data migration and the schema isn't changing?  Or should I just run this fix by hand rather than in the context of a south migration.

Output:
You can't use methods from models.py in south migrations. The reason is that in the future models.py will evolve and sooner or later you will delete those methods, then migration will be broken.
You should put all code needed by migration in migration file itself.


Input:
Python: Easily access deeply nested dict (get and set) I'm building some Python code to read and manipulate deeply nested dicts (ultimately for interacting with JSON services, however it would be great to have for other purposes)  I'm looking for a way to easily read/set/update values deep within the dict, without needing a lot of code.
@see also http://stackoverflow.com/questions/3031219/python-recursively-access-dict-via-attributes-as-well-as-index-access -- Curt Hagenlocher's "DotDictify" solution is pretty eloquent.  I also like what Ben Alman presents for JavaScript in http://benalman.com/projects/jquery-getobject-plugin/  It would be great to somehow combine the two.
Building off of Curt Hagenlocher and Ben Alman's examples, it would be great in Python to have a capability like:
>>> my_obj = DotDictify()
>>> my_obj.a.b.c = {'d':1, 'e':2}
>>> print my_obj
{'a': {'b': {'c': {'d': 1, 'e': 2}}}}
>>> print my_obj.a.b.c.d
1
>>> print my_obj.a.b.c.x
None
>>> print my_obj.a.b.c.d.x
None
>>> print my_obj.a.b.c.d.x.y.z
None

Any idea if this is possible, and if so, how to go about modifying the DotDictify solution?
Alternatively, the get method could be made to accept a dot notation (and a complementary set method added) however the object notation sure is cleaner.
>>> my_obj = DotDictify()
>>> my_obj.set('a.b.c', {'d':1, 'e':2})
>>> print my_obj
{'a': {'b': {'c': {'d': 1, 'e': 2}}}}
>>> print my_obj.get('a.b.c.d')
1
>>> print my_obj.get('a.b.c.x')
None
>>> print my_obj.get('a.b.c.d.x')
None
>>> print my_obj.get('a.b.c.d.x.y.z')
None

This type of interaction would be great to have for dealing with deeply nested dicts.  Does anybody know another strategy (or sample code snippet/library) to try?

Output:
Attribute Tree
The problem with your first specification is that Python can't tell in __getitem__ if, at my_obj.a.b.c.d, you will next proceed farther down a nonexistent tree, in which case it needs to return an object with a __getitem__ method so you won't get an AttributeError thrown at you, or if you want a value, in which case it needs to return None.
I would argue that in every case you have above, you should expect it to throw a KeyError instead of returning None.  The reason being that you can't tell if None means "no key" or "someone actually stored None at that location".  For this behavior, all you have to do is take dotdictify, remove marker, and replace __getitem__ with:
def __getitem__(self, key):
    return self[key]

Because what you really want is a dict with __getattr__ and __setattr__.
There may be a way to remove __getitem__ entirely and say something like __getattr__ = dict.__getitem__, but I think this may be over-optimization, and will be a problem if you later decide you want __getitem__ to create the tree as it goes like dotdictify originally does, in which case you would change it to:
def __getitem__(self, key):
    if key not in self:
        dict.__setitem__(self, key, dotdictify())
    return dict.__getitem__(self, key)

I don't like the marker business in the original dotdictify.
Path Support
The second specification (override get() and set()) is that a normal dict has a get() that operates differently from what you describe and doesn't even have a set (though it has a setdefault() which is an inverse operation to get()).  People expect get to take two parameters, the second being a default if the key isn't found.
If you want to extend __getitem__ and __setitem__ to handle dotted-key notation, you'll need to modify doctictify to:
class dotdictify(dict):
    def __init__(self, value=None):
        if value is None:
            pass
        elif isinstance(value, dict):
            for key in value:
                self.__setitem__(key, value[key])
        else:
            raise TypeError, 'expected dict'

    def __setitem__(self, key, value):
        if '.' in key:
            myKey, restOfKey = key.split('.', 1)
            target = self.setdefault(myKey, dotdictify())
            if not isinstance(target, dotdictify):
                raise KeyError, 'cannot set "%s" in "%s" (%s)' % (restOfKey, myKey, repr(target))
            target[restOfKey] = value
        else:
            if isinstance(value, dict) and not isinstance(value, dotdictify):
                value = dotdictify(value)
            dict.__setitem__(self, key, value)

    def __getitem__(self, key):
        if '.' not in key:
            return dict.__getitem__(self, key)
        myKey, restOfKey = key.split('.', 1)
        target = dict.__getitem__(self, myKey)
        if not isinstance(target, dotdictify):
            raise KeyError, 'cannot get "%s" in "%s" (%s)' % (restOfKey, myKey, repr(target))
        return target[restOfKey]

    def __contains__(self, key):
        if '.' not in key:
            return dict.__contains__(self, key)
        myKey, restOfKey = key.split('.', 1)
        target = dict.__getitem__(self, myKey)
        if not isinstance(target, dotdictify):
            return False
        return restOfKey in target

    def setdefault(self, key, default):
        if key not in self:
            self[key] = default
        return self[key]

    __setattr__ = __setitem__
    __getattr__ = __getitem__

Test code:
>>> life = dotdictify({'bigBang': {'stars': {'planets': {}}}})
>>> life.bigBang.stars.planets
{}
>>> life.bigBang.stars.planets.earth = { 'singleCellLife' : {} }
>>> life.bigBang.stars.planets
{'earth': {'singleCellLife': {}}}
>>> life['bigBang.stars.planets.mars.landers.vikings'] = 2
>>> life.bigBang.stars.planets.mars.landers.vikings
2
>>> 'landers.vikings' in life.bigBang.stars.planets.mars
True
>>> life.get('bigBang.stars.planets.mars.landers.spirit', True)
True
>>> life.setdefault('bigBang.stars.planets.mars.landers.opportunity', True)
True
>>> 'landers.opportunity' in life.bigBang.stars.planets.mars
True
>>> life.bigBang.stars.planets.mars
{'landers': {'opportunity': True, 'vikings': 2}}


Input: Is there a way in Python to return value via output parameter? Is there a way in Python to return a value via output parameter?

Output:
There is no reason to, since Python can return multiple values:
def func():
    return 1,2,3

a,b,c = func()

But you can also pass a mutable parameter, and return values as well:
def func(a):
    a.append(1)
    a.append(2)
    a.append(3)

L=[]
func(L)
print L


Input:
Extract files from zip without keeping the structure using python ZipFile? I try to extract all files from .zip containing subfolders in one folder.  I want all the files from subfolders extract in only one folder without keeping the original structure. At the moment, I extract all, move the files to a folder, then remove previous subfolders.  The files with same names are overwrited.
Is it possible to do it before writing files?
Here is a structure for example:
my_zip/file1.txt
my_zip/dir1/file2.txt
my_zip/dir1/dir2/file3.txt
my_zip/dir3/file4.txt

At the end I whish this:
my_dir/file1.txt
my_dir/file2.txt
my_dir/file3.txt
my_dir/file4.txt

What can I add to this code ?
import zipfile
my_dir = "D:\\Download\\"
my_zip = "D:\\Download\\my_file.zip"

zip_file = zipfile.ZipFile(my_zip, 'r')
for files in zip_file.namelist():
    zip_file.extract(files, my_dir)
zip_file.close()

if I rename files path from zip_file.namelist(), I have this error:
KeyError: "There is no item named 'file2.txt' in the archive"

Output:
This opens file handles of members of the zip archive, extracts the filename and copies it to a target file (that's how ZipFile.extract works, without taken care of subdirectories).
import os
import shutil
import zipfile

my_dir = r"D:\Download"
my_zip = r"D:\Download\my_file.zip"

with zipfile.ZipFile(my_zip) as zip_file:
    for member in zip_file.namelist():
        filename = os.path.basename(member)
        # skip directories
        if not filename:
            continue

        # copy file (taken from zipfile's extract)
        source = zip_file.open(member)
        target = file(os.path.join(my_dir, filename), "wb")
        with source, target:
            shutil.copyfileobj(source, target)


Input:
Python CSV DictReader with UTF-8 data AFAIK, the Python (v2.6) csv module can't handle unicode data by default, correct? In the Python docs there's an example on how to read from a UTF-8 encoded file. But this example only returns the CSV rows as a list.
I'd like to access the row columns by name as it is done by csv.DictReader but with UTF-8 encoded CSV input file.
Can anyone tell me how to do this in an efficient way? I will have to process CSV files in 100's of MByte in size.

Output:
Actually, I came up with an answer myself (sorry for replying to my own question):
def UnicodeDictReader(utf8_data, **kwargs):
    csv_reader = csv.DictReader(utf8_data, **kwargs)
    for row in csv_reader:
        yield {key: unicode(value, 'utf-8') for key, value in row.iteritems()}


Input:
In Python how will you multiply individual elements of an array with a floating point or integer number? S=[22 33 45.6 21.6 51.8]
P=2.45

Here S is an array
How will I multiply this and get the value?
SP=[53.9 80.85 111.72 52.92 126.91]

Output:
In numpy it is quite simple
import numpy as np
P=2.45
S=[22, 33, 45.6, 21.6, 51.8]
SP = P*np.array(S)

I recommend taking a look at the numpy tutorial for an explanation of the full capabilities of numpy's arrays:
http://www.scipy.org/Tentative_NumPy_Tutorial


Input:
Scala equivalent to Python returning multiple items In Python it's possible to do something like this:
def blarg():
    return "blargidy", "blarg"

i, j = blargh()

Is there something similar available in scala?

Output:
You can return a tuple:
def blarg = ("blargidy", "blarg")

val (i, j) = blarg

Note the pattern-matching syntax for parallel variable assignment: this works for any pattern, not just for tuples. So for instance:
val list = 1 :: 2 :: 3 :: Nil

val x :: y = list // x = 1 and y = 2 :: 3 :: Nil


Input:
Custom tab completion in python argparse Is it possible to get smarter tab completion cooperating with argparse in python scripts?  My shell is bash.  The information I can find about this stuff online is over a year old and mostly relates to optparse, which I don't want to use (deprecated).
For the following script:
#!/usr/bin/env python

import argparse as ap

def main(**args):
    pass

if __name__ == '__main__':
    parser = ap.ArgumentParser()
    parser.add_argument('positional', choices=['spam', 'eggs'])
    parser.add_argument('--optional', choices=['foo1', 'foo2', 'bar'])
    args = parser.parse_args()
    main(**vars(args))

After chmod +x on the file, my expected usage would be something like:
./example.py sp<tab>
   ->  completes to ./example.py spam
./example.py --op<tab>
   ->  completes to ./example.py --optional
./example.py --optional b<tab>
   ->  completes to ./example.py --optional bar
./example.py --optional f<tab>
   ->  completes to ./example.py --optional foo
       and further, prints  foo1  foo2  on stdout

I have no idea where to even start attacking on this ...

Output:
Have a look at argcomplete by Andrey Kislyuk.
Install it with:
sudo pip install argcomplete

Import the module and add one line in your source before calling parser.parse_args():
#!/usr/bin/env python

import argparse as ap
import argcomplete

def main(**args):
  pass

if __name__ == '__main__':
  parser = ap.ArgumentParser()
  parser.add_argument('positional', choices=['spam', 'eggs'])
  parser.add_argument('--optional', choices=['foo1', 'foo2', 'bar'])
  argcomplete.autocomplete(parser)
  args = parser.parse_args()
  main(**vars(args))

and to make sure that bash knows about this script, you use
eval "$(register-python-argcomplete your_script)"

you should put that line in your ~/.bashrc or follow argcomplete's docs and activate 'global' completion.
After that you completion works as requested.
The way this work is that the eval line creates a function _python_argcomlete which is registered using complete. (Run register-python-argcomplete your_script to just have a look at what gets eval-ed into bash). The autocomplete function looks for environment variables set by the bash completion mechanism to see if it needs to act. If it acts, it exits the program. If it doesn't act, this is a normal call to the program that function does nothing and the normal flow of the program continues.


Input:
View pdf image in an iPython Notebook The following code allows me to view a png image in an iPython notebook. Is there a way to view pdf image? I don't need to use IPython.display necessarily. I am looking for a way to print a pdf image in a file to the iPython notebook output cell.
## This is for an `png` image
from IPython.display import Image

fig = Image(filename=('./temp/my_plot.png'))
fig

Thank you.

Output:
The problem you (and others) face is that PDFs cannot be displayed directly in the browser.
The only possible way to get something similar is to use an image-converter to create a PNG or JPG out of the PDF and display this one.
This could be done via imagemagick and a custom display function.
Update 1
A simple solution is to use wand (http://docs.wand-py.org) a python-imagemagick binding. I tried with Ubuntu 13.04:

In text form:
from wand.image import Image as WImage
img = WImage(filename='hat.pdf')
img

For a multi-page pdf, you can get e.g. the second page via:
img = WImage(filename='hat.pdf[1]')

Update 2
As recent browsers support to display pdfs with their embedded pdf viewer a possible alternative solution based on an iframe can be implemented as
class PDF(object):
  def __init__(self, pdf, size=(200,200)):
    self.pdf = pdf
    self.size = size

  def _repr_html_(self):
    return '<iframe src={0} width={1[0]} height={1[1]}></iframe>'.format(self.pdf, self.size)

  def _repr_latex_(self):
    return r'\includegraphics[width=1.0\textwidth]{{{0}}}'.format(self.pdf)

This class implements html and latex representations, hence the pdf will also survive a nbconversion to latex. It can be used like
PDF('hat.pdf',size=(300,250))

With Firefox 33 this results in


Input:
FastCGI C++ vs. A Script Language (PHP/Python/Perl) What are the ups and downs of using FastCGI C++  vs.  PHP/Python/Perl to do the same job.
Any performance or design pitfalls or using one over the other? Even your opinions are welcome. (Tell me why one or the other rocks, or one or the other sucks).

Output:
scripting languages may be slower than C, but is this a problem? almost never. and if the performance becomes a problem, you start to translate only the critical parts.
twitter/ruby is a good example; ruby is slow. some of the language features (that make ruby nice in the first place) just prevent different kinds of optimization (there is a great article by the jruby guy about this ... was it ola bini? can't remember).
still, twitter is powered by ruby, because ruby is fast enough. not long ago, "the blogs" reported twitter migrating to scala for performance reasons ... the truth was, only the messaging queue (and other parts of the backend) moved to scala. yahoo runs on a mixture of languages; php for the frontend, other, faster languages are used where performance is critical.
so, why is performance not that important? there are several reasons:

database bottleneck: not the scripting is slow, the database is
clientside bottleneck: rendering in the browser takes longer than the request. optimize the server side, and nobody will notice
horizontal scaling: often it's cheaper to add another server and thus triple the requests/sec than to optimize the app
developer time and maintenance are the most expensive parts of your project. you'll get more cheap python devs that maintain your app than web-enabled c-coders in less time
no compiling, short dev cycles

another pro-scripting point: many of the scripting languages support inlining or inclusion of fast (C) code:

python, inline c
php: extensions in c
server-side javascript via rhino: direct access to java/jvm (a good example for this is orf.at, one of the biggest websites in austria, powered by helma - serverside jvm-interpreted javascript!)

i think, especially in web developement the pros of high-level scripting far outweight the cons.


Input:
Python-daemon doesn't kill its kids When using python-daemon, I'm creating subprocesses likeso:
import multiprocessing

class Worker(multiprocessing.Process):
   def __init__(self, queue):
      self.queue = queue # we wait for things from this in Worker.run()

   ...

q = multiprocessing.Queue()

with daemon.DaemonContext():
    for i in xrange(3):
       Worker(q)

    while True: # let the Workers do their thing
       q.put(_something_we_wait_for())

When I kill the parent daemonic process (i.e. not a Worker) with a Ctrl-C or SIGTERM, etc., the children don't die. How does one kill the kids?
My first thought is to use atexit to kill all the workers, likeso:
 with daemon.DaemonContext():
    workers = list()
    for i in xrange(3):
       workers.append(Worker(q))

    @atexit.register
    def kill_the_children():
        for w in workers:
            w.terminate()

    while True: # let the Workers do their thing
       q.put(_something_we_wait_for())

However, the children of daemons are tricky things to handle, and I'd be obliged for thoughts and input on how this ought to be done.
Thank you.

Output:
Your options are a bit limited. If doing self.daemon = True in the constructor for the Worker class does not solve your problem and trying to catch signals in the Parent (ie, SIGTERM, SIGINT) doesn't work, you may have to try the opposite solution - instead of having the parent kill the children, you can have the children commit suicide when the parent dies.
The first step is to give the constructor to Worker the PID of the parent process (you can do this with os.getpid()). Then, instead of just doing self.queue.get() in the worker loop, do something like this:
waiting = True
while waiting:
    # see if Parent is at home
    if os.getppid() != self.parentPID:
        # woe is me! My Parent has died!
        sys.exit() # or whatever you want to do to quit the Worker process
    try:
        # I picked the timeout randomly; use what works
        data = self.queue.get(block=False, timeout=0.1)
        waiting = False
    except queue.Queue.Empty:
        continue # try again
# now do stuff with data

The solution above checks to see if the parent PID is different than what it originally was (that is, if the child process was adopted by init or lauchd because the parent died) - see reference. However, if that doesn't work for some reason you can replace it with the following function (adapted from here):
def parentIsAlive(self):
    try:
        # try to call Parent
        os.kill(self.parentPID, 0)
    except OSError:
        # *beeep* oh no! The phone's disconnected!
        return False
    else:
        # *ring* Hi mom!
        return True

Now, when the Parent dies (for whatever reason), the child Workers will spontaneously drop like flies - just as you wanted, you daemon! :-D


Input:
Erase whole array Python How do I erase a whole array, leaving it with no items?
I want to do this so I can store new values in it (a new set of 100 floats) and find the minimum.
Right now my program is reading the minimum from sets before I think because it is appending itself with the previous set still in there.  I use .append by the way.

Output:
Note that list and array are different classes.  You can do:
del mylist[:]

This will actually modify your existing list.  David's answer creates a new list and assigns it to the same variable.  Which you want depends on the situation (e.g. does any other variable have a reference to the same list?).
Try:
a = [1,2]
b = a
a = []

and
a = [1,2]
b = a
del a[:]

Print a and b each time to see the difference.


Input:
python, format string I am trying to build a format string with lazy argument, eg I need smth like:
"%s \%s %s" % ('foo', 'bar') # "foo %s bar"

how can i do this?

Output:
"%s %%s %s" % ('foo', 'bar')

you need %%


Input:
format ints into string of hex I need to create a string of hex digits from a list of random integers (0-255). Each hex digit should be represented by two characters: 5 - "05", 16 - "10"...
e.g.

input: [0,1,2,3,127,200,255],
output: 000102037fc8ff

I've managed to come up with:
#!/usr/bin/env python

def format_me(nums):
    result = ""
    for i in nums:
        if i <= 9:
            result += "0%x" % i
        else:
            result += "%x" % i
    return result

print format_me([0,1,2,3,127,200,255])

Looks a bit awkward. Is there a simpler way?

Output:
Just for completeness, using the modern .format() syntax:
>>> numbers = [1, 15, 255]
>>> ''.join('{:02X}'.format(a) for a in numbers)
'010FFF'


Input:
switching keys and values in a dictionary in python Say I have a dictionary like so:
my_dict = {2:3, 5:6, 8:9}

Is there a way that I can switch the keys and values to get:
{3:2, 6:5, 9:8}

Output:
my_dict2 = dict((y,x) for x,y in my_dict.iteritems())

If you are using python 2.7 or 3.x you can use a dictionary comprehension instead:
my_dict2 = {y:x for x,y in my_dict.iteritems()}

Edit
As noted in the comments by JBernardo, for python 3.x you need to use items instead of iteritems


Input:
What is a generative method? I'm familiar with Python generators, however I've just come across the term "generative method" which I am not familiar with and cannot find a satisfactory definition.
To put it in context, I found the term in SQLAlchemy's narrative documentation:

Full control of the âautocommitâ behavior is available using the generative Connection.execution_options() method provided on Connection, Engine, Executable, using the âautocommitâ flag which will turn on or off the autocommit for the selected scope.

What is a generative method? Trying to iterate the object returned by Connection.execution_options() doesn't work so I'm figuring it's something other than a standard generator.

Output:
It doesn't appear to be a common database concept, but SQLAlchemy uses the term generative in the sense "generated by your program iteratively at runtime". (So, no connection to python generators). An example from the tutorial:

The Query object is fully generative, meaning that most method calls
  return a new Query object upon which further criteria may be added.
  For example, to query for users named âedâ with a full name of âEd
  Jonesâ, you can call filter() twice, which joins criteria using AND:

>>> for user in session.query(User).\
...   filter(User.name=='ed').\
...   filter(User.fullname=='Ed Jones'):
...     print user

This call syntax is more commonly known as "method chaining", and the design that allows it as a "fluent interface".
So, in the case of Connection.execution_options(), "generative" means that it returns the modified connection object, so that you can chain the calls as above.


Input: How to generate audio from a numpy array? I want to create "heart rate monitor" effect from a 2D array in numpy and want the tone to reflect the values in the array.

Output:
You can use the write function from scipy.io.wavfile to create a wav file which you can then play however you wish. Note that the array must be integers, so if you have floats, you might want to scale them appropriately:
import numpy as np
from scipy.io.wavfile import write

data = np.random.uniform(-1,1,44100) # 44100 random samples between -1 and 1
scaled = np.int16(data/np.max(np.abs(data)) * 32767)
write('test.wav', 44100, scaled)

If you want Python to actually play audio, then this page provides an overview of some of the packages/modules.


Input:
how can I set the last modified time of a file from python? I have a python script that downloads a file over FTP using ftplib.
My current download code looks just like the example in the ftp lib docs:
ftp.retrbinary('RETR README', open('README', 'wb').write)

Now I have a requirement that the file downloaded over FTP needs to have the same last modified time as the file on the FTP server itself. Assuming I could parse out the time from ftp.retrlines('list'), how can I set the modified time on the downloaded file?
I'm on a unix based OS if that matters.

Output: If you want to do this directly from python, you're looking for os.utime. The docs can give you more info.


Input:
Parsing apache log files I just started learning Python and would like to read an Apache log file and put parts of each line into different lists.
line from the file

172.16.0.3 - - [25/Sep/2002:14:04:19 +0200] "GET / HTTP/1.1" 401 - "" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020827"

according to Apache website the format is

%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\

I'm able to open the file and just read it as it is but I don't know how to make it read in that format so I can put each part in a list.

Output:
This is a job for regular expressions.
For example:
line = '172.16.0.3 - - [25/Sep/2002:14:04:19 +0200] "GET / HTTP/1.1" 401 - "" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020827"'
regex = '([(\d\.)]+) - - \[(.*?)\] "(.*?)" (\d+) - "(.*?)" "(.*?)"'

import re
print re.match(regex, line).groups()

The output would be a tuple with 6 pieces of information from the line (specifically, the groups within parentheses in that pattern):
('172.16.0.3', '25/Sep/2002:14:04:19 +0200', 'GET / HTTP/1.1', '401', '', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020827')


Input:
Is there a direct approach to format numbers in jinja2? I need to format decimal numbers in jinja2.
When I need to format dates, I call the strftime() method in my template, like this:
{{ somedate.strftime('%Y-%m-%d') }}

I wonder if there is a similar approach to do this over numbers.
Thanks in advance!

Output:
You can do it simply like this, the Python way:
{{ '%04d' % 42 }}

{{ 'Number: %d' % variable }}

Or using that method:
{{ '%d' | format(42) }}

I personally prefer the first one since it's exactly like in Python.


Input:
Pretty print 2D Python list Is there a simple, built-in way to print a 2D Python list as a 2D matrix?
So this:
[["A", "B"], ["C", "D"]]

would become something like
A    B
C    D

I found the pprint module, but it doesn't seem to do what I want.

Output:
To make things interesting, let's try with a bigger matrix:
matrix = [
   ["Ah!",  "We do have some Camembert", "sir"],
   ["It's a bit", "runny", "sir"],
   ["Well,",  "as a matter of fact it's", "very runny, sir"],
   ["I think it's runnier",  "than you",  "like it, sir"]
]

s = [[str(e) for e in row] for row in matrix]
lens = [max(map(len, col)) for col in zip(*s)]
fmt = '\t'.join('{{:{}}}'.format(x) for x in lens)
table = [fmt.format(*row) for row in s]
print '\n'.join(table)

Output:
Ah!                     We do have some Camembert   sir
It's a bit              runny                       sir
Well,                   as a matter of fact it's    very runny, sir
I think it's runnier    than you                    like it, sir

UPD: for multiline cells, something like this should work:
text = [
    ["Ah!",  "We do have\nsome Camembert", "sir"],
    ["It's a bit", "runny", "sir"],
    ["Well,",  "as a matter\nof fact it's", "very runny,\nsir"],
    ["I think it's\nrunnier",  "than you",  "like it,\nsir"]
]

from itertools import chain, izip_longest

matrix = chain.from_iterable(
    izip_longest(
        *(x.splitlines() for x in y),
        fillvalue='')
    for y in text)

And then apply the above code.
See also http://pypi.python.org/pypi/texttable


Input:
Python exception handling - line number I'm using python to evaluate some measured data. Because of many possible results it is difficult to handle or possible combinations. Sometimes an error happens during the evaluation. It is usually an index error because I get out of range from measured data.
It is very difficult to find out on which place in code the problem happened. It would help a lot if I knew on which line the error was raised. If I use following code:
try:
    result = evaluateData(data)
except Exception, err:
    print ("Error: %s.\n" % str(err))

Unfortunately this only tells me that there is and index error. I would like to know more details about the exception (line in code, variable etc.) to find out what happened. Is it possible?
Thank you.

Output:
Solution, printing filename, linenumber, line itself and exception descrition:
import linecache
import sys

def PrintException():
    exc_type, exc_obj, tb = sys.exc_info()
    f = tb.tb_frame
    lineno = tb.tb_lineno
    filename = f.f_code.co_filename
    linecache.checkcache(filename)
    line = linecache.getline(filename, lineno, f.f_globals)
    print 'EXCEPTION IN ({}, LINE {} "{}"): {}'.format(filename, lineno, line.strip(), exc_obj)

try:
    print 1/0
except:
    PrintException()

Output:
EXCEPTION IN (D:/Projects/delme3.py, LINE 15 "print 1/0"): integer division or modulo by zero


Input:
flask sqlalchemy querying a column with not equals I can query my seats table for all seats where there is no invite assigned:
seats = Seat.query.filter_by(invite=None).all()

However, when querying for all seats that have an invite assigned, I get an err:
seats = Seat.query.filter_by(invite!=None).all()

NameError: name 'invite' is not defined

Here is my Seat class:
class Seat(db.Model):
    id = db.Column(db.Integer, primary_key=True)

    invite_id = db.Column(db.Integer, db.ForeignKey('invite.id'))
    invite = db.relationship('Invite',
        backref=db.backref('folks', lazy='dynamic'))

How can I query for all seats where the owner is not blank?
Thanks

Output:
The filter_by() method takes a sequence of keyword arguments, so you always have to use '=' with it. You want to use the filter() method which allows for !=.
seats = Seat.query.filter(Seat.invite != None).all()

or
seats = Seat.query.filter(Seat.invite_id != None).all()


Input:
u'\ufeff' in Python string I get an error with the following patter:
UnicodeEncodeError: 'ascii' codec can't encode character u'\ufeff' in position 155: ordinal not in range(128)

Not sure what u'\ufeff' is, it shows up when I'm web scraping. How can I remedy the situation? The .replace() string method doesn't work on it.

Output:
The Unicode character U+FEFF is the byte order mark, or BOM, and is used to tell the difference between big- and little-endian UTF-16 encoding.  If you decode the web page using the right codec, Python will remove it for you.  Examples:
#!python2
#coding: utf8
u = u'ABC'
e8 = u.encode('utf-8')        # encode without BOM
e8s = u.encode('utf-8-sig')   # encode with BOM
e16 = u.encode('utf-16')      # encode with BOM
e16le = u.encode('utf-16le')  # encode without BOM
e16be = u.encode('utf-16be')  # encode without BOM
print 'utf-8     %r' % e8
print 'utf-8-sig %r' % e8s
print 'utf-16    %r' % e16
print 'utf-16le  %r' % e16le
print 'utf-16be  %r' % e16be
print
print 'utf-8  w/ BOM decoded with utf-8     %r' % e8s.decode('utf-8')
print 'utf-8  w/ BOM decoded with utf-8-sig %r' % e8s.decode('utf-8-sig')
print 'utf-16 w/ BOM decoded with utf-16    %r' % e8s.decode('utf-16')
print 'utf-16 w/ BOM decoded with utf-16le  %r' % e8s.decode('utf-16le')

Note that EF BB BF is a UTF-8-encoded BOM.  It is not required for UTF-8, but serves only as a signature (usually on Windows).
Output:
utf-8     'ABC'
utf-8-sig '\xef\xbb\xbfABC'
utf-16    '\xff\xfeA\x00B\x00C\x00'    # Adds BOM and encodes using native processor endian-ness.
utf-16le  'A\x00B\x00C\x00'
utf-16be  '\x00A\x00B\x00C'

utf-8  w/ BOM decoded with utf-8     u'\ufeffABC'    # doesn't remove BOM if present.
utf-8  w/ BOM decoded with utf-8-sig u'ABC'          # removes BOM if present.
utf-16 w/ BOM decoded with utf-16    u'ABC'          # *requires* BOM to be present.
utf-16 w/ BOM decoded with utf-16le  u'\ufeffABC'    # doesn't remove BOM if present.

Note that the utf-16 coded requires BOM to be present, or Python won't know if the data is big- or little-endian.


Input:
flake8 complains on boolean comparison "==" in filter clause I have a boolean field in the mysql db table.
# table model
class TestCase(Base):
    __tablename__ = 'test_cases'
    ...
    obsoleted = Column('obsoleted',  Boolean)

To get the count of all the non-obsoleted test cases, that can be done simply like this:
caseNum = session.query(TestCase).filter(TestCase.obsoleted == False).count()
print(caseNum)

That works fine, but the flake8 report the following warning:

E712: Comparison to False should be "if cond is False:" or "if not
  cond:"

Okay, I think that make sense. So change my code to this:
caseNum = session.query(TestCase).filter(TestCase.obsoleted is False).count()

or
caseNum = session.query(TestCase).filter(not TestCase.obsoleted).count()

But neither of them can work.  The result is always 0.
I think the filter clause doesn't support the operator "is" or "is not". Will someone can tell me how to handle this situation. I don't want to disable the flake.

Output:
That's because SQLAlchemy filters are one of the few places where == False actually makes sense. Everywhere else you should not use it.
Add a # noqa comment to the line and be done with it.
Or you can use sqlalchemy.sql.expression.false:
from sqlalchemy.sql.expression import false

TestCase.obsoleted == false()

where false() returns the right value for your session SQL dialect. There is a matching sqlalchemy.expression.true.


Input:
python subprocess changing directory I want to execute a script inside a subdirectory/superdirectory (I need to be inside this sub/super-directory first). I can't get subprocess to enter my subdirectory:
tducin@localhost:~/Projekty/tests/ve$ python
Python 2.7.4 (default, Sep 26 2013, 03:20:26)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> import os
>>> os.getcwd()
'/home/tducin/Projekty/tests/ve'
>>> subprocess.call(['cd ..'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/subprocess.py", line 524, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Python throws OSError and I don't know why. It doesn't matter whether I try to go into an existing subdir or go one directory up (as above) - I always end up with the same error.

Output:
What your code tries to do is call a program named cd ... What you want is call a command named cd.
But cd is a shell internal. So you can only call it as
subprocess.call('cd ..', shell=True) # pointless code! See text below.

But it is pointless to do so. As no process can change another process's working directory (again, at least on a UNIX-like OS, but as well on Windows), this call will have the subshell change its dir and exit immediately.
What you want can be achieved with os.chdir() or with the subprocess named parameter cwd which changes the working directory immediately before executing a subprocess.


Input:
Matplotlib log scale tick label number formatting With matplotlib when a log scale is specified for an axis, the default method of labeling that axis is with numbers that are 10 to a power eg. 10^6. Is there an easy way to change all of these labels to be their full numerical representation? eg. 1, 10, 100, etc.
Note that I do not know what the range of powers will be and want to support an arbitrary range (negatives included).

Output:
Sure, just change the formatter.
For example, if we have this plot:
import matplotlib.pyplot as plt

fig, ax = plt.subplots()
ax.axis([1, 10000, 1, 100000])
ax.loglog()

plt.show()

You could set the tick labels manually, but then the tick locations and labels would be fixed when you zoom/pan/etc.  Therefore, it's best to change the formatter:
from matplotlib.ticker import ScalarFormatter
for axis in [ax.xaxis, ax.yaxis]:
    axis.set_major_formatter(ScalarFormatter())


Input:
How does the min/max function on a nested list work? Lets say, there is a nested list, like:
my_list = [[1, 2, 21], [1, 3], [1, 2]]

When the function min() is called on this:
min(my_list)

The output received is
[1, 2]

Why and How does it work? What are some use cases of it?

Output:
How are lists and other sequences compared in Python?
Lists (and other sequences) in Python are compared lexicographically and not based on any other parameter.

Sequence objects may be compared to other objects with the same sequence type. The comparison uses lexicographical ordering: first the first two items are compared, and if they differ this determines the outcome of the comparison; if they are equal, the next two items are compared, and so on, until either sequence is exhausted.

What is lexicographic sorting?
From the Wikipedia page on lexicographic sorting

lexicographic or lexicographical order (also known as lexical order, dictionary order, alphabetical order or lexicographic(al) product) is a generalization of the way the alphabetical order of words is based on the alphabetical order of their component letters.

The min function returns the smallest value in the iterable. So the lexicographic value of [1,2] is the least in that list. You can check by using [1,2,21]
>>> my_list=[[1,2,21],[1,3],[1,2]]
>>> min(my_list)
[1, 2]

What is happening in this case of min?
Going element wise on my_list, firstly [1,2,21] and [1,3]. Now from the docs

If two items to be compared are themselves sequences of the same type, the lexicographical comparison is carried out recursively.

Thus the value of [1,1,21] is less than [1,3], because the second element of [1,3], which is, 3 is lexicographically higher than the value of the second element of [1,1,21], which is, 1.
Now comparing [1,2] and [1,2,21], and adding another reference from the docs

If one sequence is an initial sub-sequence of the other, the shorter sequence is the smaller (lesser) one.

[1,2] is an initial sub-sequence of [1,2,21]. Therefore the value of [1,2] on the whole is smaller than that of [1,2,21]. Hence [1,2] is returned as the output.
This can be validated by using the sorted function
>>> sorted(my_list)
[[1, 2], [1, 2, 21], [1, 3]]

What if the list has multiple minimum elements?
If the list contains duplicate min elements the first is returned
>>> my_list=[[1,2],[1,2]]
>>> min(my_list)
[1, 2]

This can be confirmed using the id function call
>>> my_list=[[1,2],[1,2]]
>>> [id(i) for i in my_list]
[140297364849368, 140297364850160]
>>> id(min(my_list))
140297364849368

What do I need to do to prevent lexicographic comparison in min?
If the required comparison is not lexicographic then the key argument can be used (as mentioned by Padraic)
The min function has an additional optional argument called key. The key argument takes a function.

The optional key argument specifies a one-argument ordering function
  like that used for list.sort(). The key argument, if supplied, must be
  in keyword form (for example, min(a,b,c,key=func)).

For example, if we need the smallest element by length, we need to use the len function.
>>> my_list=[[1,2,21],[1,3],[1,2]]
>>> min(my_list,key=len)            # Notice the key argument
[1, 3]

As we can see the first shortest element is returned here.

What if the list is heterogeneous?
Until Python2
If the list is heterogeneous type names are considered for ordering, check Comparisions,

Objects of different types except numbers are ordered by their type names

Hence if you put an int and a list there you will get the integer value as the smallest as i is of lower value than l. Similarly '1' would be of higher value than both of this.
>>> my_list=[[1,1,21],1,'1']
>>> min(my_list)
1

Python3 and onwards
However this confusing technique was removed in Python3. It now raises a TypeError. Read What's new in Python 3.0

The ordering comparison operators (<, <=, >=, >) raise a TypeError exception when the operands donât have a meaningful natural ordering. Thus, expressions like 1 < '', 0 > None or len <= len are no longer valid, and e.g. None < None raises TypeError instead of returning False. A corollary is that sorting a heterogeneous list no longer makes sense â all the elements must be comparable to each other.

>>> my_list=[[1,1,21],1,'1']
>>> min(my_list)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unorderable types: int() < list()

But it works for Comparable types, For example
>>> my_list=[1,2.0]
>>> min(my_list)
1

Here we can see that the list contains float values and int values. But as float and int are comparable types, min function works in this case.


Input:
py2exe + sqlalchemy + sqlite problem I am playing around with getting some basic stuff to work in Python before i go into full speed dev mode. Here are the specifics:
Python 2.5.4
PyQt4 4.4.3
SqlAlchemy 0.5.2
py2exe 0.6.9
setuptools 0.6c9
pysqlite 2.5.1

setup.py:
from distutils.core import setup
import py2exe

setup(windows=[{"script" : "main.py"}], options={"py2exe" : {"includes" : ["sip", "PyQt4.QtSql","sqlite3"],"packages":["sqlite3",]}})

py2exe appears to generate the .exe file correctly, but when i execute dist/main.exe i get this in the main.exe.log
Traceback (most recent call last):
  File "main.py", line 18, in <module>
  File "main.py", line 14, in main
  File "db\manager.pyc", line 12, in __init__
  File "sqlalchemy\engine\__init__.pyc", line 223, in create_engine
  File "sqlalchemy\engine\strategies.pyc", line 48, in create
  File "sqlalchemy\engine\url.pyc", line 91, in get_dialect
ImportError: No module named sqlite

I've been googling my heart out, but can't seem to find any solutions to this. If i can't get this to work now, my hopes of using Python for this project will be dashed and i will start over using Ruby... (not that there is anything wrong with Ruby, i just wanted to use this project as a good way to teach myself Python)

Output:
you need to include the sqlalchemy.databases.sqlite package
setup(
  windows=[{"script" : "main.py"}],
  options={"py2exe" : {
    "includes": ["sip", "PyQt4.QtSql"],
    "packages": ["sqlalchemy.databases.sqlite"]
}})


Input:
How to execute a process remotely using python I want to connect too and execute a process on a remote server using python.  I want to be able to get the return code and stderr(if any) of the process.  Has anyone ever done anything like this before.  I have done it with ssh, but I want to do it from python script.
Cheers.

Output:
Use the ssh module called paramiko which was created for this purpose instead of using subprocess. Here's an example below:
from paramiko import SSHClient
client = SSHClient()
client.load_system_host_keys()
client.connect("hostname", username="user")
stdin, stdout, stderr = client.exec_command('program')
print "stderr: ", stderr.readlines()
print "pwd: ", stdout.readlines()

UPDATE: The example used to use the ssh module, but that is now deprecated and paramiko is the up-to-date module that provides ssh functionality in python.


Input:
Delete multiple files matching a pattern I have made an online gallery using Python and Django. I've just started to add editing functionality, starting with a rotation. I use sorl.thumbnail to auto-generate thumbnails on demand.
When I edit the original file, I need to clean up all the thumbnails so new ones are generated. There are three or four of them per image (I have different ones for different occasions).
I could hard-code in the file-varients... But that's messy and if I change the way I do things, I'll need to revisit the code.
Ideally I'd like to do a regex-delete. In regex terms, all my originals are named like so:
^(?P<photo_id>\d+)\.jpg$

So I want to delete:
^(?P<photo_id>\d+)[^\d].*jpg$

(Where I replace photo_id with the ID I want to clean.)

Output:
Try something like this:
import os, re

def purge(dir, pattern):
    for f in os.listdir(dir):
    	if re.search(pattern, f):
    		os.remove(os.path.join(dir, f))

Then you would pass the directory containing the files and the pattern you wish to match.


Input:
Python parsing bracketed blocks What would be the best way in Python to parse out chunks of text contained in matching brackets?
"{ { a } { b } { { { c } } } }"

should initially return:
[ "{ a } { b } { { { c } } }" ]

putting that as an input should return:
[ "a", "b", "{ { c } }" ]

which should return:
[ "{ c }" ]

[ "c" ]

[]

Output:
Or this pyparsing version:
>>> from pyparsing import nestedExpr
>>> txt = "{ { a } { b } { { { c } } } }"
>>>
>>> nestedExpr('{','}').parseString(txt).asList()
[[['a'], ['b'], [[['c']]]]]
>>>


Input:
How to apply itertools.product to elements of a list of lists? I have a list of arrays and I would like to get the cartesian product of the elements in the arrays.
I will use an example to make this more concrete...
itertools.product seems to do the trick but I am stuck in a little detail.
arrays = [(-1,+1), (-2,+2), (-3,+3)];

If I do
cp = list(itertools.product(arrays));

I get
cp = cp0 = [((-1, 1),), ((-2, 2),), ((-3, 3),)]

But what I want to get is
cp1 = [(-1,-2,-3), (-1,-2,+3), (-1,+2,-3), (-1,+2,+3), ..., (+1,+2,-3), (+1,+2,+3)].

I have tried a few different things:
cp = list(itertools.product(itertools.islice(arrays, len(arrays))));
cp = list(itertools.product(iter(arrays, len(arrays))));

They all gave me cp0 instead of cp1.
Any ideas?
Thanks in advance.

Output:
>>> arrays = [(-1,+1), (-2,+2), (-3,+3)]
>>> list(itertools.product(*arrays))
[(-1, -2, -3), (-1, -2, 3), (-1, 2, -3), (-1, 2, 3), (1, -2, -3), (1, -2, 3), (1, 2, -3), (1, 2, 3)]


Input: Plotting implicit equations in 3d I'd like to plot implicit equation F(x,y,z) = 0 in 3D. Is it possible in Matplotlib?

Output:
You can trick matplotlib into plotting implicit equations in 3D.  Just make a one-level contour plot of the equation for each z value within the desired limits.  You can repeat the process along the y and z axes as well for a more solid-looking shape.
from mpl_toolkits.mplot3d import axes3d
import matplotlib.pyplot as plt
import numpy as np

def plot_implicit(fn, bbox=(-2.5,2.5)):
    ''' create a plot of an implicit function
    fn  ...implicit function (plot where fn==0)
    bbox ..the x,y,and z limits of plotted interval'''
    xmin, xmax, ymin, ymax, zmin, zmax = bbox*3
    fig = plt.figure()
    ax = fig.add_subplot(111, projection='3d')
    A = np.linspace(xmin, xmax, 100) # resolution of the contour
    B = np.linspace(xmin, xmax, 15) # number of slices
    A1,A2 = np.meshgrid(A,A) # grid on which the contour is plotted

    for z in B: # plot contours in the XY plane
        X,Y = A1,A2
        Z = fn(X,Y,z)
        cset = ax.contour(X, Y, Z+z, [z], zdir='z')
        # [z] defines the only level to plot for this contour for this value of z

    for y in B: # plot contours in the XZ plane
        X,Z = A1,A2
        Y = fn(X,y,Z)
        cset = ax.contour(X, Y+y, Z, [y], zdir='y')

    for x in B: # plot contours in the YZ plane
        Y,Z = A1,A2
        X = fn(x,Y,Z)
        cset = ax.contour(X+x, Y, Z, [x], zdir='x')

    # must set plot limits because the contour will likely extend
    # way beyond the displayed level.  Otherwise matplotlib extends the plot limits
    # to encompass all values in the contour.
    ax.set_zlim3d(zmin,zmax)
    ax.set_xlim3d(xmin,xmax)
    ax.set_ylim3d(ymin,ymax)

    plt.show()

Here's the plot of the Goursat Tangle:
def goursat_tangle(x,y,z):
    a,b,c = 0.0,-5.0,11.8
    return x**4+y**4+z**4+a*(x**2+y**2+z**2)**2+b*(x**2+y**2+z**2)+c

plot_implicit(goursat_tangle)

You can make it easier to visualize by adding depth cues with creative colormapping:

Here's how the OP's plot looks:
def hyp_part1(x,y,z):
    return -(x**2) - (y**2) + (z**2) - 1

plot_implicit(hyp_part1, bbox=(-100.,100.))

Bonus:  You can use python to functionally combine these implicit functions:
def sphere(x,y,z):
    return x**2 + y**2 + z**2 - 2.0**2

def translate(fn,x,y,z):
    return lambda a,b,c: fn(x-a,y-b,z-c)

def union(*fns):
    return lambda x,y,z: np.min(
        [fn(x,y,z) for fn in fns], 0)

def intersect(*fns):
    return lambda x,y,z: np.max(
        [fn(x,y,z) for fn in fns], 0)

def subtract(fn1, fn2):
    return intersect(fn1, lambda *args:-fn2(*args))

plot_implicit(union(sphere,translate(sphere, 1.,1.,1.)), (-2.,3.))


Input:
Internals of Python list, access and resizing runtimes Is Python's [] a list or array. Is the access time of an index O(1) like an array or O(n) like a list, and is appending/resizing O(1) like a list or O(n) like an array, or is it a hybrid that can manage O(1) for accessing and resizing?
I read somewhere (http://norvig.com/python-lisp.html) that array access is really slow in Python. However, when I wrote a memoized version of a recursive fibonacci procedure using both a dictionary (Python's dictionary is suppose to be really fast) and list, they had equal times. Why is this? Does a Python tuple have faster access times than a python list?

Output:
Python's [] is implemented as an array, not a linked list. Although resizing is O(n), appending to it is amortized O(1), because resizes happen very rarely. If you're not familiar with how this works, read this Wikipedia entry on dynamic arrays. Python's list doesn't expand by a factor of 2 each time, it's a bit more complicated than that, but the expansions are still designed make appending amortized O(1).
Inserting in the middle, however, is always an inefficient O(n), because n items may have to be moved.
Tuples aren't faster than lists - they're just immutable lists under the hood (*).
Regarding your dictionary test: depending on your exact implementation, caching in a list will be faster than with a dict. However, Python's dicts are highly optimized, and especially for small amounts of keys will perform great.

(*) Here's a list's "get item" C function in Python 2.6:
PyObject *
PyList_GetItem(PyObject *op, Py_ssize_t i)
{
    if (!PyList_Check(op)) {
        PyErr_BadInternalCall();
        return NULL;
    }
    if (i < 0 || i >= Py_SIZE(op)) {
        if (indexerr == NULL)
            indexerr = PyString_FromString(
                "list index out of range");
        PyErr_SetObject(PyExc_IndexError, indexerr);
        return NULL;
    }
    return ((PyListObject *)op) -> ob_item[i];
}

And this is a tuple's:
PyObject *
PyTuple_GetItem(register PyObject *op, register Py_ssize_t i)
{
    if (!PyTuple_Check(op)) {
        PyErr_BadInternalCall();
        return NULL;
    }
    if (i < 0 || i >= Py_SIZE(op)) {
        PyErr_SetString(PyExc_IndexError, "tuple index out of range");
        return NULL;
    }
    return ((PyTupleObject *)op) -> ob_item[i];
}

As you can see, they're almost exactly the same. In the end, after some type and bound checking, it's a simple pointer access with an index.


Input:
Why can't generators be pickled? Python's pickle (I'm talking standard Python 2.5/2.6/2.7 here) cannot pickle locks, file objects etc.
It also cannot pickle generators and lambda expressions (or any other anonymous code), because the pickle really only stores name references.
In case of locks and OS-dependent features, the reason why you cannot pickle them is obvious and makes sense.
But why can't you pickle generators?

Note: just for clarity -- I'm interested in the fundamental reason (or assumptions and choices that went into that design decision) why, not in "because it gives you a Pickle error".
I realize the question's a bit wide-aimed, so here's a rule of thumb of whether your answered it: "If these assumptions were raised, or the type of allowed generator somehow more restricted, would pickling generators work again?"

Output:
There is lots of information about this available. For the "official word" on the issue, read the (closed) Python bugtracker issue.
The core reasoning, by one of the people who made the decision, is detailed on this blog:

Since a generator is essentially a souped-up function, we would need to save its bytecode, which is not guarantee to be backward-compatible between Pythonâs versions, and its frame, which holds the state of the generator such as local variables, closures and the instruction pointer. And this latter is rather cumbersome to accomplish, since it basically requires to make the whole interpreter picklable. So, any support for pickling generators would require a large number of changes to CPythonâs core.
Now if an object unsupported by pickle (e.g., a file handle, a socket, a database connection, etc) occurs in the local variables of a generator, then that generator could not be pickled automatically, regardless of any pickle support for generators we might implement. So in that case, you would still need to provide custom __getstate__ and __setstate__ methods. This problem renders any pickling support for generators rather limited.

And two suggested workarounds are mentioned:

Anyway, if you need for a such feature, then look into Stackless Python which does all the above. And since Stacklessâs interpreter is picklable, you also get process migration for free. This means you can interrupt a tasklet (the name for Stacklessâs green threads), pickle it, send the pickle to a another machine, unpickle it, resume the tasklet, and voilÃ  youâve just migrated a process. This is freaking cool feature!
But in my humble opinion, the best solution to this problem to the rewrite the generators as simple iterators (i.e., one with a __next__ method). Iterators are easy and efficient space-wise to pickle because their state is explicit. You would still need to handle objects representing some external state explicitly however; you cannot get around this.


Input:
Create (sane/safe) filename from any (unsafe) string I want to create a sane/safe filename (i.e. somewhat readable, no "strange" characters, etc.) from some random Unicode string (mich might contain just anything).
(It doesn't matter for me wether the function is Cocoa, ObjC, Python, etc.)

Of course, there might be infinite many characters which might be strange. Thus, it is not really a solution to have a blacklist and to add more and more to that list over the time.
I could have a whitelist. However, I don't really know how to define it. [a-zA-Z0-9 .] is a start but I also want to accept unicode chars which can be displayed in a normal way.

Output:
Python:
"".join([c for c in filename if c.isalpha() or c.isdigit() or c==' ']).rstrip()

this accepts Unicode characters but removes line breaks, etc.
example:
filename = u"ad\nbla'{-+\)(Ã§?"

gives: adblaÃ§
edit
str.isalnum() does alphanumeric on one step. â comment from queueoverflow below. danodonovan hinted on keeping a dot included.
    keepcharacters = (' ','.','_')
    "".join(c for c in filename if c.isalnum() or c in keepcharacters).rstrip()


Input:
python class instance variables and class variables im having a problem understanding how class / instance variables work in python. I dont understand why when i try this code the list variable seems to be a class variable
class testClass():
    list = []
    def __init__(self):
        self.list.append('thing')

p = testClass()
print p.list

f = testClass()
print f.list

output:
['thing']
['thing', 'thing']

and when i do this it seems to be an instance variable
class testClass():
    def __init__(self):
        self.list = []
        self.list.append('thing')

p = testClass()
print p.list

f = testClass()
print f.list

output:
['thing']
['thing']

many thanks
jon

Output:
This is because of the way Python resolves names with the .. When you write self.list the Python runtime tries to resolve the list name first looking for it in the instance object, and if not found in the class instance.
Let's look into it step by step
self.list.append(1)

Is there a list name into the object self?

Yes: Use it! Finish.
No: Go to 2.

Is there a list name into the class instance of object self?

Yes: Use it! Finish
No: Error!

But when you bind a name things are different:
self.list = []

Is there a list name into the object self?

Yes: Overwrite it!
No: Bind it!

So, that is always an instance variable.
Your first example creates a list into the class instance, as this is the active scope at the time (no self anywhere). But your second example creates a list explicitly in the scope of self.
More interesting would be the example:
class testClass():
    list = ['foo']
    def __init__(self):
        self.list = []
        self.list.append('thing')

x = testClass()
print x.list
print testClass.list
del x.list
print x.list

That will print:
['thing']
['foo']
['foo']

The moment you delete the instance name the class name is visible through the self reference.


Input:
What does '@reify' do and when should it be used? I saw it in the Pyramid tutorial for UX design. I couldn't make out much what this decorator is all about.
Sample code where I saw its usage.
def __init__(self, request):
    self.request = request
    renderer = get_renderer("templates/global_layout.pt")
    self.global_template = renderer.implementation().macros['layout']

@reify
def company_name(self):
    return COMPANY

@reify
def site_menu(self):
    new_menu = SITE_MENU[:]
    url = self.request.url
    for menu in new_menu:
        if menu['title'] == 'Home':
            menu['current'] = url.endswith('/')
        else:
            menu['current'] = url.endswith(menu['href'])
    return new_menu

@view_config(renderer="templates/index.pt")
def index_view(self):
    return {"page_title": "Home"}

@view_config(renderer="templates/about.pt", name="about.html")
def about_view(self):
    return {"page_title": "About"}

Output:
From the source code documentation:

""" Put the result of a method which uses this (non-data)
      descriptor decorator in the instance dict after the first call,
      effectively replacing the decorator with an instance variable."""

A description from from the fuzzy notepad blog sums it up nicely.

It acts like @property, except that the function is only ever called
  once; after that, the value is cached as a regular attribute. This
  gives you lazy attribute creation on objects that are meant to be
  immutable.

So in the code you posted, site_menu can be accesses like a cached property.


Input:
Check if an object exists I need to check if Model.objects.filter(...) turned up anything, but do not need to insert anything. My code so far is:
user_pass = log_in(request.POST)  # form class
if user_pass.is_valid():
    cleaned_info = user_pass.cleaned_data
    user_object = User.objects.filter(email = cleaned_info['username'])

Output:
I think the easiest from a logical and efficiency point of view is using the queryset's exists() function, documented here:
https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.exists
So in your example above I would simply write:
if User.objects.filter(email = cleaned_info['username']).exists():
    # at least one object satisfying query exists
else:
    # no object satisfying query exists


Input:
Is the PySide Slot Decorator Necessary? I've seen some example code for PySide slots that uses the @QtCore.Slot decorator, and some that does not.  Testing it myself, it doesn't seem to make a difference.  Is there a reason I should or should not use it?  For example, in the following code:
import sys
from PySide import QtCore

# the next line seems to make no difference
@QtCore.Slot()
def a_slot(s):
    print s

class SomeClass(QtCore.QObject):
    happened = QtCore.Signal(str)
    def __init__(self):
        QtCore.QObject.__init__(self)
    def do_signal(self):
        self.happened.emit("Hi.")

sc = SomeClass()
sc.happened.connect(a_slot)
sc.do_signal()

the @QtCore.Slot decorator makes no difference; I can omit it, call @QtCore.Slot(str), or even @QtCore.Slot(int), and it still nicely says, "Hi."
The same seems to be true for PyQt's pyqtSlot.

Output:
This link explains the following about the pyqtSlot decorator:

Although PyQt4 allows any Python callable to be used as a slot when
  connecting signals, it is sometimes necessary to explicitly mark a
  Python method as being a Qt slot and to provide a C++ signature for
  it. PyQt4 provides the pyqtSlot() function decorator to do this.

and

Connecting a signal to a decorated Python method also has the
  advantage of reducing the amount of memory used and is slightly
  faster.

Since the pyqtSlot decorator can take additional argument such as name, it allows different Python methods to handle the different signatures of a signal.
If you don't use the slot decorator, the signal connection mechanism has to manually work out all the type conversions to map from the underlying C++ function signatures to the Python functions.  When the slot decorators are used, the type mapping can be explicit.