Repository: pandas-dev/pandas
Pull request: updating df.query and df.eval docstrings. resolves #16283
- [x] closes #16283
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This is a docstring improvement suggested in #16283 issue discussion. `df.eval` and `df.query` are updated for consistency and adding exemples with `@` usage for local variables

Problem statement:
df.eval does not see globals
#### Code Sample, a copy-pastable example if possible

```python
from numpy import pi
import pandas as pd

df = pd.DataFrame({'a': [1, 2], 'b': [3, 4]})

print('pi' in globals())
print(df.eval('a * b * pi'))
```
#### Problem description
This raises a 
```
pandas.computation.ops.UndefinedVariableError: name 'pi' is not defined
```

According to the documentation of `pd.eval` which is used by `pd.DataFrame.eval`, 
globals should be available:

> local_dict : dict or None, optional
A dictionary of local variables, taken from locals() by default.
global_dict : dict or None, optional
A dictionary of global variables, taken from globals() by default.

The same exception is raised even if `global_dict=globals()` is passed to eval explicitly.

The only solution I found was to use:

```python
pd.eval('a * b * pi', resolvers=[df, globals()])
```

#### Expected Output
No exception and the result of the calculation

#### Output of ``pd.show_versions()``

<details>
# Paste the output here pd.show_versions() here

INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.8.0-51-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 35.0.1
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.19.0
statsmodels: 0.8.0
xarray: None
IPython: 6.0.0
sphinx: 1.4.8
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: 1.2.0
tables: 3.3.0
numexpr: 2.6.2
matplotlib: 2.0.1
openpyxl: 2.4.1
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.3
bs4: 4.5.3
html5lib: 0.999
httplib2: 0.9.2
apiclient: None
sqlalchemy: 1.1.9
pymysql: 0.7.2.None
psycopg2: None
jinja2: 2.9.6
boto: 2.43.0
pandas_datareader: Non
</details>

Hints from issue discussion:
Variables outside the df context should be prefixed with `@` - see docs [here](http://pandas.pydata.org/pandas-docs/stable/enhancingperf.html#local-variables) - although it'd be good to add an example to the docstring too.

```python
In [74]: df.eval('a * b * @pi')
Out[74]: 
0     9.424778
1    25.132741
dtype: float64
```

Commit message: adding 1 example

Patch to review:
diff --git a/pandas/core/frame.py b/pandas/core/frame.py
index f72a214f120a0..a8db983f8e61e 100644
--- a/pandas/core/frame.py
+++ b/pandas/core/frame.py
@@ -4604,6 +4604,14 @@ def query(self, expr: str, *, inplace: bool = False, **kwargs) -> DataFrame | No
         >>> df[df.B == df["C C"]]
            A   B  C C
         0  1  10   10
+
+        Using local variable:
+
+        >>> local_var = 2
+        >>> df.query("A <= @local_var")
+        A   B  C C
+        0  1  10   10
+        1  2   8    9
         """
         inplace = validate_bool_kwarg(inplace, "inplace")
         if not isinstance(expr, str):
@@ -4644,6 +4652,20 @@ def eval(self, expr: str, *, inplace: bool = False, **kwargs) -> Any | None:
         ----------
         expr : str
             The expression string to evaluate.
+
+            You can refer to variables
+            in the environment by prefixing them with an '@' character like
+            ``@a + b``.
+
+            You can refer to column names that are not valid Python variable names
+            by surrounding them in backticks. Thus, column names containing spaces
+            or punctuations (besides underscores) or starting with digits must be
+            surrounded by backticks. (For example, a column named "Area (cm^2)" would
+            be referenced as ```Area (cm^2)```). Column names which are Python keywords
+            (like "list", "for", "import", etc) cannot be used.
+
+            For example, if one of your columns is called ``a a`` and you want
+            to sum it with ``b``, your query should be ```a a` + b``.
         inplace : bool, default False
             If the expression contains an assignment, whether to perform the
             operation inplace and mutate the existing DataFrame. Otherwise,
@@ -4675,14 +4697,16 @@ def eval(self, expr: str, *, inplace: bool = False, **kwargs) -> Any | None:
 
         Examples
         --------
-        >>> df = pd.DataFrame({"A": range(1, 6), "B": range(10, 0, -2)})
+        >>> df = pd.DataFrame(
+        ...     {"A": range(1, 6), "B": range(10, 0, -2), "C C": range(10, 5, -1)}
+        ... )
         >>> df
-           A   B
-        0  1  10
-        1  2   8
-        2  3   6
-        3  4   4
-        4  5   2
+           A   B  C C
+        0  1  10   10
+        1  2   8    9
+        2  3   6    8
+        3  4   4    7
+        4  5   2    6
         >>> df.eval("A + B")
         0    11
         1    10
@@ -4695,19 +4719,19 @@ def eval(self, expr: str, *, inplace: bool = False, **kwargs) -> Any | None:
         modified.
 
         >>> df.eval("C = A + B")
-           A   B   C
-        0  1  10  11
-        1  2   8  10
-        2  3   6   9
-        3  4   4   8
-        4  5   2   7
+           A   B  C C   C
+        0  1  10   10  11
+        1  2   8    9  10
+        2  3   6    8   9
+        3  4   4    7   8
+        4  5   2    6   7
         >>> df
-           A   B
-        0  1  10
-        1  2   8
-        2  3   6
-        3  4   4
-        4  5   2
+           A   B  C C
+        0  1  10   10
+        1  2   8    9
+        2  3   6    8
+        3  4   4    7
+        4  5   2    6
 
         Multiple columns can be assigned to using multi-line expressions:
 
@@ -4717,12 +4741,32 @@ def eval(self, expr: str, *, inplace: bool = False, **kwargs) -> Any | None:
         ... D = A - B
         ... '''
         ... )
-           A   B   C  D
-        0  1  10  11 -9
-        1  2   8  10 -6
-        2  3   6   9 -3
-        3  4   4   8  0
-        4  5   2   7  3
+           A   B  C C   C  D
+        0  1  10   10  11 -9
+        1  2   8    9  10 -6
+        2  3   6    8   9 -3
+        3  4   4    7   8  0
+        4  5   2    6   7  3
+
+        For columns with spaces in their name, you can use backtick quoting.
+
+        >>> df.eval("B * `C C`")
+        0    100
+        1     72
+        2     48
+        3     28
+        4     12
+
+        Local variables shall be explicitly referenced using ``@``
+        character in front of the name:
+
+        >>> local_var = 2
+        >>> df.eval("@local_var * A")
+        0     2
+        1     4
+        2     6
+        3     8
+        4    10
         """
         from pandas.core.computation.eval import eval as _eval
 


Code review comments:
diff_hunk: @@ -4695,19 +4719,19 @@ def eval(self, expr: str, *, inplace: bool = False, **kwargs) -> Any | None:
         modified.
 
         >>> df.eval("C = A + B")
-           A   B   C
-        0  1  10  11
-        1  2   8  10
-        2  3   6   9
-        3  4   4   8
-        4  5   2   7
+           A   B  C C   C
path: pandas/core/frame.py
text: @user1:
These headers don't look correct anymore

@author:
I double checked and it looks correct

@user1:
If you look at the examples here you can see 5 column labels for 4 columns: https://pandas.pydata.org/preview/pandas-dev/pandas/58749/docs/reference/api/pandas.DataFrame.eval.html 

@author:
Initial example dataframe contains 3 columns: `A`, `B` and `C C`. The eval method will add a fourth column named `C` in the resulting dataframe for this example.
Repr of the dataframe does not make the separation of columns obvious when columns contains spaces. It is like there is 3 columns `C` when you actually have `C C` and `C`.

To make the example cristal clear I may change `C C` by `C&C` and update the examples to add columns with names differents than `C`.

Do you want me to update also `query` examples as `C C` is actually coming from there ??

@user1:
> To make the example cristal clear I may change C C by C&C and update the examples to add columns with names differents than C.

Ah yes that would be great. Sorry I didn't backtrack to see the context of the original columns

@author:
shall I update `query` as well ?

@user1:
Yes please