Repository: truera/trulens
Issue: OpenAI error on BasicApp
Root cause: Validation Gap

I'm testing out monitoring using the TruBasicApp.

I've already implemented a pretty complex call on my Windows PC, but when trying to upload it to a Linux-based server on AWS it is raising an error with the `OpenAIEndpoint`, that raises at the *costs* section.

This is the code I'm trying to run:

```py
import os
import dotenv

from trulens_eval import Tru, TruBasicApp, Feedback
from trulens_eval.schema import Cost, Record

SCRIPT_DIR = os.path.abspath(os.path.dirname(__file__))
dotenv.load_dotenv(os.path.join(SCRIPT_DIR, "my.env"))

tru = Tru(database_url=os.environ.get("database_url"))

def llm_standalone(prompt):
    return prompt

recorder = TruBasicApp(llm_standalone, app="default")

with recorder as recording:
    response = recorder.app("Hola pa")

print(response)
```

As I said, I tried keeping it as simple as I could, and it was working for me on my Windows PC, but on the AWS setup I got this error:

```
Traceback (most recent call last):
  File "/path/to/my/file/trulens.py", line 125, in <module>
    tru.add_record(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/app.py", line 700, in __exit__
    raise exc_value
  File "/path/to/my/file/trulens.py", line 115, in <module>
    response = recorder.app(prompt, max_tokens, stop_sequences)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/tru_basic_app.py", line 51, in __call__
    return self._call(*args, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/instruments.py", line 783, in wrapper
    ctx.app._on_add_record(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/app.py", line 748, in _on_add_record
    raise error
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/instruments.py", line 734, in wrapper
    rets, cost = Endpoint.track_all_costs_tally(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 374, in track_all_costs_tally
    result, cbs = Endpoint.track_all_costs(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 265, in track_all_costs
    e = OpenAIEndpoint()
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/openai.py", line 212, in __init__
    self._instrument_module_members(openai, "create")
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 242, in _instrument_module_members
    self._instrument_class(obj, method_name=method_name)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 227, in _instrument_class
    if hasattr(cls, method_name):
  File "/opt/conda/lib/python3.9/site-packages/openai/_utils/_proxy.py", line 22, in __getattr__
    return getattr(self.__get_proxied__(), attr)
  File "/opt/conda/lib/python3.9/site-packages/openai/_utils/_proxy.py", line 43, in __get_proxied__
    return self.__load__()
  File "/opt/conda/lib/python3.9/site-packages/openai/_module_client.py", line 36, in __load__
    return _load_client().audio
  File "/opt/conda/lib/python3.9/site-packages/openai/__init__.py", line 311, in _load_client
    _client = _ModuleClient(
  File "/opt/conda/lib/python3.9/site-packages/openai/_client.py", line 93, in __init__
    raise OpenAIError(
openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
```

I tried searching for the error but I didn't know how to continue ate the `_instrument_class` method.

Some other things I tried:
- Since I'm not actually using openAI services, I tried changing the [this boolean]() as a *kwarg* of the recorder call like `recorder.app(.., with_openai=False)` call, but I got an error in some point forward saying `with_openai` is not a vallid rgument:
```
Traceback (most recent call last):
  File "/path/to/my/file/trulens.py", line 23, in <module>
    response = recorder.app("Hola pa", with_openai=False)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/app.py", line 700, in __exit__
    raise exc_value
  File "/path/to/my/file/trulens.py", line 23, in <module>
    response = recorder.app("Hola pa", with_openai=False)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/tru_basic_app.py", line 51, in __call__
    return self._call(*args, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/instruments.py", line 783, in wrapper
    ctx.app._on_add_record(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/app.py", line 743, in _on_add_record
    record = ctx.finish_record(build_record)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/app.py", line 382, in finish_record
    record = calls_to_record(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/app.py", line 725, in build_record
    main_in = self.main_input(func, sig, bindings)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/tru_basic_app.py", line 151, in main_input
    bindings = sig.bind(*bindings.args[1:], **bindings.kwargs)
  File "/opt/conda/lib/python3.9/inspect.py", line 3045, in bind
    return self._bind(args, kwargs)
  File "/opt/conda/lib/python3.9/inspect.py", line 3034, in _bind
    raise TypeError(
TypeError: got an unexpected keyword argument 'with_openai'
```
- Setting an empty value on the `OPENAI_API_KEY` environment variable solves that error but raises another forward on the code:
```
Traceback (most recent call last):
  File "/path/to/my/file/trulens.py", line 121, in <module>
    tru.add_record(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/app.py", line 700, in __exit__
    raise exc_value
  File "/path/to/my/file/trulens.py", line 111, in <module>
    response = recorder.app(prompt, max_tokens, stop_sequences)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/tru_basic_app.py", line 51, in __call__
    return self._call(*args, **kwargs)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/instruments.py", line 783, in wrapper
    ctx.app._on_add_record(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/app.py", line 748, in _on_add_record
    raise error
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/instruments.py", line 734, in wrapper
    rets, cost = Endpoint.track_all_costs_tally(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 374, in track_all_costs_tally
    result, cbs = Endpoint.track_all_costs(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 265, in track_all_costs
    e = OpenAIEndpoint()
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/openai.py", line 212, in __init__
    self._instrument_module_members(openai, "create")
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 242, in _instrument_module_members
    self._instrument_class(obj, method_name=method_name)
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 229, in _instrument_class
    f"Instrumenting {cls.__name__}.{method_name} for {self.name}"
  File "/opt/conda/lib/python3.9/site-packages/openai/_utils/_proxy.py", line 22, in __getattr__
    return getattr(self.__get_proxied__(), attr)
AttributeError: 'Completions' object has no attribute '__name__'
```
- Finally, to see if it had smth to do with the Linux context, I tried this same code in Collab. The weird part there is that the first time I run it, the same error is raised, but if I run it once more without killing the kernel, it works properly (I really don't know what is happening under the hood). Since I'm running this code on a Python script I can't keep a kernel alive, but I did the closest thing which was catching the error with a `try.. except..` and running the process again on the *except* codeblock. It did work but not consistently since the record was sometimes kept and sometimes not so it would generate problems later.

I don't know what else to do, maybe I'm missing something really simple here..

Hi @lopezjuanma96 ; I'm investigating this and I see there are related issues that arise from newer openai package versions. Can you share with us the list of packages and their versions that are installed i.e. `pip list` ?

Hey! For sure:

```
Package                       Version
----------------------------- ---------------
aiobotocore                   2.5.0
aiofiles                      22.1.0
aiohttp                       3.8.5
aioitertools                  0.7.1
aiosignal                     1.2.0
aiosqlite                     0.18.0
aiostream                     0.5.2
alabaster                     0.7.12
alembic                       1.12.1
altair                        5.1.2
anaconda-anon-usage           0.4.2
anaconda-catalogs             0.2.0
anaconda-client               1.12.1
anaconda-cloud-auth           0.1.3
anaconda-navigator            2.5.0
anaconda-project              0.11.1
anthropic                     0.5.0
anthropic-bedrock             0.4.0
anyio                         3.5.0
appdirs                       1.4.4
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
arrow                         1.2.3
astroid                       2.14.2
astropy                       5.3.4
asttokens                     2.0.5
async-timeout                 4.0.2
atomicwrites                  1.4.0
attrs                         23.1.0
Automat                       20.2.0
autopep8                      1.6.0
awswrangler                   3.4.0
Babel                         2.11.0
backcall                      0.2.0
backoff                       2.2.1
backports.functools-lru-cache 1.6.4
backports.tempfile            1.0
backports.weakref             1.0.post1
bcrypt                        3.2.0
beautifulsoup4                4.12.2
binaryornot                   0.4.4
black                         0.0
bleach                        4.1.0
blinker                       1.6.3
bokeh                         3.2.1
boltons                       23.0.0
boto3                         1.28.82
botocore                      1.31.82
Bottleneck                    1.3.5
Brotli                        1.0.9
brotlipy                      0.7.0
cachetools                    5.3.2
certifi                       2023.7.22
cffi                          1.15.1
chardet                       4.0.0
charset-normalizer            2.0.4
click                         8.1.7
cloudpickle                   2.2.1
clyent                        1.2.2
cohere                        4.32
colorama                      0.4.6
colorcet                      3.0.1
comm                          0.1.2
conda                         23.7.4
conda-build                   3.26.1
conda-content-trust           0.2.0
conda_index                   0.3.0
conda-libmamba-solver         23.7.0
conda-pack                    0.6.0
conda-package-handling        2.2.0
conda_package_streaming       0.9.0
conda-repo-cli                1.0.75
conda-token                   0.4.0
conda-verify                  3.4.2
constantly                    15.1.0
contourpy                     1.0.5
cookiecutter                  1.7.3
cryptography                  41.0.3
cssselect                     1.1.0
cycler                        0.11.0
cytoolz                       0.12.0
daal4py                       2023.1.1
dask                          2023.6.0
dataclasses-json              0.5.14
datasets                      2.12.0
datashader                    0.15.2
datashape                     0.5.4
debugpy                       1.6.7
decorator                     5.1.1
defusedxml                    0.7.1
Deprecated                    1.2.14
diff-match-patch              20200713
dill                          0.3.6
distributed                   2023.6.0
distro                        1.8.0
dnspython                     2.4.2
docopt                        0.6.2
docstring-to-markdown         0.11
docutils                      0.18.1
entrypoints                   0.4
et-xmlfile                    1.1.0
executing                     0.8.3
Faker                         19.12.0
fastavro                      1.8.2
fastjsonschema                2.16.2
favicon                       0.7.0
filelock                      3.9.0
flake8                        6.0.0
Flask                         2.2.2
fonttools                     4.25.0
frozendict                    2.3.8
frozenlist                    1.4.0
fsspec                        2023.10.0
future                        0.18.3
gensim                        4.3.0
gitdb                         4.0.11
GitPython                     3.1.40
glob2                         0.7
greenlet                      2.0.1
h11                           0.14.0
h5py                          3.9.0
HeapDict                      1.0.1
holoviews                     1.17.1
htbuilder                     0.6.2
httpcore                      1.0.1
httpx                         0.25.1
huggingface-hub               0.15.1
humanize                      4.8.0
hvplot                        0.8.4
hyperlink                     21.0.0
idna                          3.4
imagecodecs                   2023.1.23
imageio                       2.31.4
imagesize                     1.4.1
imbalanced-learn              0.10.1
importlib-metadata            6.0.0
incremental                   21.3.0
inflection                    0.5.1
iniconfig                     1.1.1
intake                        0.6.8
intervaltree                  3.1.0
ipykernel                     6.25.0
ipython                       8.15.0
ipython-genutils              0.2.0
ipywidgets                    8.0.4
isort                         5.9.3
itemadapter                   0.3.0
itemloaders                   1.0.4
itsdangerous                  2.0.1
jaraco.classes                3.2.1
jedi                          0.18.1
jellyfish                     1.0.1
Jinja2                        3.1.2
jinja2-time                   0.2.0
jmespath                      0.10.0
joblib                        1.2.0
json5                         0.9.6
jsonpatch                     1.33
jsonpointer                   2.1
jsonschema                    4.19.2
jsonschema-specifications     2023.7.1
jupyter                       1.0.0
jupyter_client                7.4.9
jupyter-console               6.6.3
jupyter_core                  5.5.0
jupyter-events                0.8.0
jupyter-server                1.23.4
jupyter_server_fileid         0.9.0
jupyter_server_ydoc           0.8.0
jupyter-ydoc                  0.2.4
jupyterlab                    3.6.3
jupyterlab-pygments           0.1.2
jupyterlab_server             2.22.0
jupyterlab-widgets            3.0.5
kaggle                        1.5.16
kaleido                       0.2.1
keyring                       23.13.1
kiwisolver                    1.4.4
langchain                     0.0.333
langsmith                     0.0.62
lazy_loader                   0.3
lazy-object-proxy             1.6.0
libarchive-c                  2.9
libmambapy                    1.5.3
linkify-it-py                 2.0.0
llama-index                   0.8.55
llvmlite                      0.41.0
lmdb                          1.4.1
locket                        1.0.0
loguru                        0.7.2
lxml                          4.9.3
lz4                           4.3.2
Mako                          1.2.4
Markdown                      3.4.1
markdown-it-py                2.2.0
markdownlit                   0.0.7
MarkupSafe                    2.1.1
marshmallow                   3.20.1
matplotlib                    3.7.2
matplotlib-inline             0.1.6
mccabe                        0.7.0
mdit-py-plugins               0.3.0
mdurl                         0.1.0
menuinst                      1.4.19
merkle-json                   1.0.0
millify                       0.1.1
mistune                       0.8.4
mkl-fft                       1.3.8
mkl-random                    1.2.4
mkl-service                   2.4.0
more-itertools                8.12.0
mpmath                        1.3.0
msgpack                       1.0.3
multidict                     6.0.2
multipledispatch              0.6.0
multiprocess                  0.70.14
munch                         4.0.0
munkres                       1.1.4
mypy-extensions               1.0.0
navigator-updater             0.4.0
nbclassic                     0.5.5
nbclient                      0.5.13
nbconvert                     6.5.4
nbformat                      5.9.2
nest-asyncio                  1.5.6
networkx                      3.1
nltk                          3.8.1
notebook                      6.5.4
notebook_shim                 0.2.2
numba                         0.58.0
numexpr                       2.8.7
numpy                         1.24.3
numpydoc                      1.5.0
openai                        0.28.1
openpyxl                      3.0.10
packaging                     23.1
pandas                        2.1.1
pandocfilters                 1.5.0
panel                         1.2.3
param                         1.13.0
paramiko                      2.8.1
parsel                        1.6.0
parso                         0.8.3
partd                         1.4.1
pathlib                       1.0.1
pathspec                      0.10.3
patsy                         0.5.3
pep8                          1.7.1
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        10.0.1
pinecone-client               2.2.4
pip                           23.2.1
pipreqs                       0.4.13
pkce                          1.0.3
pkginfo                       1.9.6
platformdirs                  3.10.0
plotly                        5.9.0
pluggy                        1.0.0
ply                           3.11
poyo                          0.5.0
prometheus-client             0.14.1
prompt-toolkit                3.0.36
Protego                       0.1.16
protobuf                      4.24.4
psutil                        5.9.0
psycopg2                      2.9.9
ptyprocess                    0.7.0
pure-eval                     0.2.2
py-cpuinfo                    9.0.0
pyarrow                       11.0.0
pyasn1                        0.4.8
pyasn1-modules                0.2.8
pycodestyle                   2.10.0
pycosat                       0.6.6
pycparser                     2.21
pyct                          0.5.0
pycurl                        7.45.2
pydantic                      1.10.8
pydeck                        0.8.1b0
PyDispatcher                  2.0.5
pydocstyle                    6.3.0
pyerfa                        2.0.0
pyflakes                      3.0.1
Pygments                      2.15.1
PyJWT                         2.4.0
pylint                        2.16.2
pylint-venv                   2.3.0
pyls-spyder                   0.4.0
pymdown-extensions            10.3.1
PyNaCl                        1.5.0
pyodbc                        4.0.39
pyOpenSSL                     23.2.0
pyparsing                     3.0.9
PyQt5                         5.15.10
PyQt5-sip                     12.13.0
PyQtWebEngine                 5.15.6
pyrsistent                    0.18.0
PySocks                       1.7.1
pytest                        7.4.0
pytest-subtests               0.11.0
python-dateutil               2.8.2
python-decouple               3.8
python-dotenv                 0.21.0
python-json-logger            2.0.7
python-lsp-black              1.2.1
python-lsp-jsonrpc            1.0.0
python-lsp-server             1.7.2
python-slugify                5.0.2
python-snappy                 0.6.1
pytoolconfig                  1.2.5
pytz                          2023.3.post1
pyviz-comms                   2.3.0
PyWavelets                    1.4.1
pywin32                       305.1
pywin32-ctypes                0.2.0
pywinpty                      2.0.10
PyYAML                        6.0.1
pyzmq                         23.2.0
QDarkStyle                    3.0.2
qstylizer                     0.2.2
QtAwesome                     1.2.2
qtconsole                     5.4.2
QtPy                          2.2.0
queuelib                      1.6.2
referencing                   0.30.2
regex                         2023.10.3
requests                      2.31.0
requests-file                 1.5.1
requests-toolbelt             1.0.0
responses                     0.13.3
rfc3339-validator             0.1.4
rfc3986-validator             0.1.1
rich                          13.6.0
rope                          1.7.0
rpds-py                       0.10.6
Rtree                         1.0.1
ruamel.yaml                   0.17.21
ruamel-yaml-conda             0.17.21
s3fs                          2023.4.0
s3transfer                    0.7.0
safetensors                   0.4.0
scikit-image                  0.20.0
scikit-learn                  1.3.0
scikit-learn-intelex          20230426.121932
scipy                         1.11.3
Scrapy                        2.8.0
seaborn                       0.12.2
Send2Trash                    1.8.2
service-identity              18.1.0
setuptools                    68.0.0
sip                           6.7.12
six                           1.16.0
slack-bolt                    1.18.0
slack-sdk                     3.23.0
smart-open                    5.2.1
smmap                         5.0.1
sniffio                       1.2.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
soupsieve                     2.5
Sphinx                        5.0.2
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
spyder                        5.4.3
spyder-kernels                2.4.4
SQLAlchemy                    2.0.21
st-annotated-text             4.0.1
stack-data                    0.2.0
statsmodels                   0.14.0
streamlit                     1.28.0
streamlit-aggrid              0.3.4.post3
streamlit-camera-input-live   0.2.0
streamlit-card                0.0.61
streamlit-embedcode           0.1.2
streamlit-extras              0.3.4
streamlit-faker               0.0.3
streamlit-image-coordinates   0.1.6
streamlit-javascript          0.1.5
streamlit-keyup               0.2.0
streamlit-toggle-switch       1.0.2
streamlit-vertical-slider     1.0.2
sympy                         1.11.1
tables                        3.8.0
tabulate                      0.8.10
TBB                           0.2
tblib                         1.7.0
tenacity                      8.2.2
terminado                     0.17.1
text-unidecode                1.3
textdistance                  4.2.1
threadpoolctl                 2.2.0
three-merge                   0.1.1
tifffile                      2023.4.12
tiktoken                      0.5.1
tinycss2                      1.2.1
tldextract                    3.2.0
tokenizers                    0.13.3
toml                          0.10.2
tomlkit                       0.11.1
toolz                         0.12.0
tornado                       6.3.3
tqdm                          4.65.0
traitlets                     5.7.1
transformers                  4.32.1
trulens-eval                  0.16.0
Twisted                       22.10.0
twisted-iocpsupport           1.0.2
typing_extensions             4.5.0
typing-inspect                0.8.0
tzdata                        2023.3
tzlocal                       5.2
uc-micro-py                   1.0.1
ujson                         5.4.0
Unidecode                     1.2.0
urllib3                       1.26.18
validators                    0.22.0
w3lib                         1.21.0
watchdog                      2.1.6
wcwidth                       0.2.5
webencodings                  0.5.1
websocket-client              0.58.0
Werkzeug                      2.2.3
whatthepatch                  1.0.2
wheel                         0.38.4
widgetsnbextension            4.0.5
win-inet-pton                 1.1.0
win32-setctime                1.1.0
wrapt                         1.14.1
xarray                        2023.6.0
xlwings                       0.29.1
xxhash                        2.0.2
xyzservices                   2022.9.0
y-py                          0.5.9
yapf                          0.31.0
yarg                          0.1.9
yarl                          1.8.1
ypy-websocket                 0.8.2
zict                          3.0.0
zipp                          3.11.0
zope.interface                5.4.0
zstandard                     0.19.0
```

Still consider the fact that I exported this package list from my Windows run (that was working) to the AWS environment and did `pip install -r requirements.txt` and it still raised the error, so it might have smth to do with that Linux OS, wether on how the `openai` package works or what dependencies it installs..

BTW, thanks for the quick response, and don't doubt contacting me for any further doubts

Hey @lopezjuanma96 - can you check to see if this is resolved with trulens_eval==0.18.0 and lmk if you're still having issues?

Thanks!

Hi! I'm away for the week but I'll make sure to try it out next week!

@lopezjuanma96 - just checking back on this. Let me know when you get a chance to try the fix. Thanks!

Hi josh! Sorry I was about to try this yesterday but had the busiest day! Just tested it and keep getting the same error with `trulens_eval==0.18.0`

This is the output:
```
/opt/conda/lib/python3.9/site-packages/langchain/__init__.py:34: UserWarning: Importing PromptTemplate from langchain root module is no longer supported. Please use langchain.prompts.PromptTemplate instead.
  warnings.warn(
🦑 Tru initialized with db url sqlite:///default.sqlite .
🛑 Secret keys may be written to the database. See the `database_redact_keys` option of `Tru` to prevent this.
Error calling wrapped function _call.
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/instruments.py", line 738, in wrapper
    rets, cost = Endpoint.track_all_costs_tally(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 373, in track_all_costs_tally
    result, cbs = Endpoint.track_all_costs(
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/base.py", line 264, in track_all_costs
    e = OpenAIEndpoint()
  File "/opt/conda/lib/python3.9/site-packages/trulens_eval/feedback/provider/endpoint/openai.py", line 213, in __init__
    kwargs['client'] = OpenAI()
  File "/opt/conda/lib/python3.9/site-packages/openai/_client.py", line 93, in __init__
    raise OpenAIError(
openai.OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable
```

Be sure to tell me if you need me to test out anything else!

Hi! I've been experimenting with the `language_verification` feedback function (which uses Hugging Face models, not OpenAI) in the `trulens_eval` library, and I think I'm having the same issue.

I'm using the `TruBasicApp` class and attempting to run it within a context manager. However, since the OpenAI API key is not provided, it's throwing an `openai.OpenAIError` -- even when OpenAI functionalities are not explicitly invoked.

My code:
```
from trulens_eval import TruBasicApp
translate_recorder = TruBasicApp(translate_app, app_id="translate-app1", feedbacks=feedbacks)

with translate_recorder as recording: # issue here
    for prompt in prompts:
        print(prompt)
        _ = translate_recorder.app(prompt)
        
tru_record = recording.records[0]
```

### (Possible) Root Cause
Upon investigation, the issue does seem to stem from the `track_all_costs_tally()` static method (in `instruments.py`). This method, in turn, calls the track_all_costs() method with the default argument `with_openai=True` (I also attempted to input this as a kwarg with no luck). 

The code should be able to handle a missing API key with the ApiKeyError exception: 

```    
def track_all_costs(
        thunk: Thunk[T],
        with_openai: bool = True,
        with_hugs: bool = True,
        with_litellm: bool = True
    ) -> Tuple[T, Sequence[EndpointCallback]]:
        """
        Track costs of all of the apis we can currently track, over the
        execution of thunk.
        """
        endpoints = []
        if with_openai:
            # TODO: DEPS
            from trulens_eval.feedback.provider.endpoint.openai import \
                OpenAIEndpoint

            try:
                e = OpenAIEndpoint()
                endpoints.append(e)
            except ApiKeyError:
                logger.debug(
                    "OpenAI API keys are not set. "
                    "Will not track usage."
                )
```

but instead, it raises openai's own `OpenAIError` exception:
```
OpenAIError("The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable")
```

After some digging, I think the problem might be in the initialization of the `OpenAIEndpoint` (`trulens_eval/feedback/provider/endpoint/openai.py`). The function `_check_key("OPENAI_API_KEY", silent=True, warn=False)`, which is responsible for raising an `ApiKeyError`, is commented out and never invoked during the initialization.


Hi; I'm having trouble reproducing (though I'm not on Windows). Will try windows soon. Can you share the variables names in your `my.env` ? 

@saradiazdelser Looking at one of the issues you pointed out, I made a possible fix but unsure whether it solves the whole problem: https://github.com/truera/trulens/pull/627 .


Hi @piotrm0! Thank you for your response!!
I ran it again with the possible fix.  It's no longer throwing the OpenaiAPI Key error, but it's raising a different one:

```
Error calling wrapped function _call.
Traceback (most recent call last):
  File "/home/sdiaz/Projects/trulens/trulens_eval/trulens_eval/instruments.py", line 738, in wrapper
    rets, cost = Endpoint.track_all_costs_tally(
  File "/home/sdiaz/Projects/trulens/trulens_eval/trulens_eval/feedback/provider/endpoint/base.py", line 383, in **track_all_costs_tally**
    result, cbs = Endpoint.track_all_costs(
  File "/home/sdiaz/Projects/trulens/trulens_eval/trulens_eval/feedback/provider/endpoint/base.py", line 309, in track_all_costs
    return Endpoint._track_costs(thunk, with_endpoints=endpoints)
  File "/home/sdiaz/Projects/trulens/trulens_eval/trulens_eval/feedback/provider/endpoint/base.py", line 486, in _track_costs
    result: T = thunk()
  File "/home/sdiaz/Projects/trulens/trulens_eval/trulens_eval/instruments.py", line 739, in <lambda>
    lambda: func(*bindings.args, **bindings.kwargs)
  File "/home/sdiaz/Projects/trulens/trulens_eval/trulens_eval/tru_basic_app.py", line 48, in _call
    return self._call_fn(*args, **kwargs)
  File "/tmp/ipykernel_95178/1271406272.py", line 15, in translate_app
    llm_response = chain(prompt)
  File "/home/sdiaz/Projects/trulens/.venv/lib/python3.9/site-packages/langchain/chains/base.py", line 312, in __call__
    raise e
  File "/home/sdiaz/Projects/trulens/.venv/lib/python3.9/site-packages/langchain/chains/base.py", line 306, in __call__
    self._call(inputs, run_manager=run_manager)
  File "/home/sdiaz/Projects/trulens/.venv/lib/python3.9/site-packages/langchain/chains/llm.py", line 103, in _call
    response = self.generate([inputs], run_manager=run_manager)
  File "/home/sdiaz/Projects/trulens/.venv/lib/python3.9/site-packages/langchain/chains/llm.py", line 115, in generate
    return self.llm.generate_prompt(
  File "/home/sdiaz/Projects/trulens/.venv/lib/python3.9/site-packages/langchain_core/language_models/llms.py", line 506, in generate_prompt
...
    raise NotImplementedError(
NotImplementedError: text/html output type is not implemented yet. You can pass `raw_response=True` to get the raw `Response` object and parse the output by yourself.

```


@saradiazdelser is this still ongoing?

Seems like this issue is stale. @saradiazdelser - please reopen if this is ongoing.