Q:

Remove an open file if an error occurs

Is it possible to close and delete while using 'with open()'?  
I will occasionally encounter an error while doing calculations/extractions/queries in a routine called 'write_file'.  
try:
    with open(some_file, 'w') as report:
        write_file(report, other_variables)
except:
    logging.error("Report {} did not compile".format(some_file))

I wrapped this in a try/except, but it still wrote the report up to the exception. 

A:

If you're comfortable deleting the file after encountering any exception at all, then this will suffice:
import os

try:
    with open(some_file, 'w') as report:
        write_file(report, other_variables)
except:
    logging.error("Report {} did not compile".format(some_file))
    os.remove(some_file)

Just keep in mind that it's almost always better to be specific about the exception you're catching. 
Some free advice: if I were concerned about writing nonsense to a file, I would separate what you're doing into two distinct steps. 
First, I would determine prior to even opening the file to being written whether or not some calculation or statement will throw an exception.  If it does, I wouldn't even bother opening the file. 
Second, if the first step passes without an exception, I would open up the file and write to it.  You can optionally wrap this step around a try/except block to catch file IO errors. 
The benefit of splitting up your work like this is that it makes it easier to diagnose an issue should one occur.  The class of exceptions resulting from the first step are bound to be distinct from the class of exceptions that would result from the second step.