Repository: Lightning-AI/pytorch-lightning

Issue:
Use .comet.config file for CometLogger
## 🚀 Feature

When creating a CometML experiment normally, the API key will be read from the file `~/.comet.config` or from an environment variable if it isn't passed in directly. It would be nice if the `CometLogger` supported these uses as well.

### Motivation

Putting the API key in code is certainly a bad practice, and it's a pain to have to export it as an environment variable and then get its value in Python or else read it from the file manually. Adding this feature makes things more seamless compared to how people use CometML when not using PyTorch Lightning.

### Additional context

I have a [patch] written for this already; it only changes a few lines of code. From the template message when I went to create a PR, though, it seemed I should create an issue first. Let me know if you have thoughts about this.

(somewhat related - the REST API key is also [deprecated]; the normal API key should be used instead now. I didn't change that code, though, because I'm not sure if older versions of Comet would have any issues with that change.)

[patch]: https://github.com/PyTorchLightning/pytorch-lightning/compare/master...neighthan:master
[deprecated]: https://www.comet.ml/docs/python-sdk/advanced/#python-configuration


Discussion:

Resolving patch:
diff --git a/pytorch_lightning/loggers/comet.py b/pytorch_lightning/loggers/comet.py
--- a/pytorch_lightning/loggers/comet.py
+++ b/pytorch_lightning/loggers/comet.py
@@ -16,8 +16,7 @@
     except ImportError:  # pragma: no-cover
         # For more information, see: https://www.comet.ml/docs/python-sdk/releases/#release-300
         from comet_ml.papi import API  # pragma: no-cover
-
-    _COMET_AVAILABLE = True
+    from comet_ml.config import get_config, get_api_key
 except ImportError:  # pragma: no-cover
     CometExperiment = None
     CometExistingExperiment = None
@@ -25,6 +24,8 @@
     CometBaseExperiment = None
     API = None
     _COMET_AVAILABLE = False
+else:
+    _COMET_AVAILABLE = True
 
 
 import torch
@@ -78,8 +79,11 @@ class CometLogger(LightningLoggerBase):
         >>> trainer = Trainer(logger=comet_logger)
 
     Args:
-        api_key: Required in online mode. API key, found on Comet.ml
-        save_dir: Required in offline mode. The path for the directory to save local comet logs
+        api_key: Required in online mode. API key, found on Comet.ml. If not given, this
+            will be loaded from the environment variable COMET_API_KEY or ~/.comet.config
+            if either exists.
+        save_dir: Required in offline mode. The path for the directory to save local
+            comet logs. If given, this also sets the directory for saving checkpoints.
         workspace: Optional. Name of workspace for this user
         project_name: Optional. Send your experiment to a specific project.
             Otherwise will be sent to Uncategorized Experiments.
@@ -88,6 +92,10 @@ class CometLogger(LightningLoggerBase):
             This is used to determine version number
         experiment_name: Optional. String representing the name for this particular experiment on Comet.ml.
         experiment_key: Optional. If set, restores from existing experiment.
+        offline: If api_key and save_dir are both given, this determines whether
+            the experiment will be in online or offline mode. This is useful if you use
+            save_dir to control the checkpoints directory and have a ~/.comet.config
+            file but still want to run offline experiments.
     """
 
     def __init__(self,
@@ -98,6 +106,7 @@ def __init__(self,
                  rest_api_key: Optional[str] = None,
                  experiment_name: Optional[str] = None,
                  experiment_key: Optional[str] = None,
+                 offline: bool = False,
                  **kwargs):
 
         if not _COMET_AVAILABLE:
@@ -105,10 +114,15 @@ def __init__(self,
                               ' install it with `pip install comet-ml`.')
         super().__init__()
         self._experiment = None
-        self._save_dir = save_dir
 
         # Determine online or offline mode based on which arguments were passed to CometLogger
-        if api_key is not None:
+        api_key = api_key or get_api_key(None, get_config())
+
+        if api_key is not None and save_dir is not None:
+            self.mode = "offline" if offline else "online"
+            self.api_key = api_key
+            self._save_dir = save_dir
+        elif api_key is not None:
             self.mode = "online"
             self.api_key = api_key
         elif save_dir is not None:
@@ -116,7 +130,9 @@ def __init__(self,
             self._save_dir = save_dir
         else:
             # If neither api_key nor save_dir are passed as arguments, raise an exception
-            raise MisconfigurationException("CometLogger requires either api_key or save_dir during initialization.")
+            raise MisconfigurationException(
+                "CometLogger requires either api_key or save_dir during initialization."
+            )
 
         log.info(f"CometLogger will be initialized in {self.mode} mode")
 


Test patch: