Repository: skypilot-org/skypilot
Pull request: [Spot] Rename spot queue and fix help str
Fixes #1277 and #1195.

Main changes:
1. Rename `sky spot status` to `sky spot queue`, and deprecate `sky spot status`.
```
> sky spot --help                                                   
Usage: sky spot [OPTIONS] COMMAND [ARGS]...

  Managed spot instances related commands.

Options:
  -h, --help  Show this message and exit.

Commands:
  launch  Launch a managed spot job.
  queue   Show managed spot job queue.
  cancel  Cancel managed spot jobs.
  logs    Tail the log of a managed spot job.
```
```
> sky spot status                                                                   
WARNING: `status` is deprecated and will be removed in the future release. Please use `queue` instead.

Fetching managed spot job statuses...
Managed spot jobs:
In progress jobs: 2 RUNNING, 1 STARTING

ID  NAME                            RESOURCES        SUBMITTED   TOT. DURATION  JOB DURATION  #RECOVERIES  STATUS     
11  test-log                        1x [CPU:0.5]     1 min ago   1m 57s         -             0            STARTING   
10  test-log                        1x [CPU:0.5]     6 mins ago  6m 23s         3m 46s        0            RUNNING    
9   test-log                        1x [CPU:0.5]     1 hr ago    1h 22m 11s     1h 19m 42s    0            RUNNING    
8   test-logging                    1x [CPU:0.5]     12 hrs ago  3h 1m 45s      2h 56m 26s    1            CANCELLED  
7   sky-08bc-zhwu                   1x [CPU:0.5]     1 day ago   2m 28s         1s            0            SUCCEEDED  
6   test-spot-recovery-9ce1ce58-8c  1x [CPU:0.5]     2 days ago  9m 53s         4m 2s         1            CANCELLED  
5   test-spot-storage-9ce1ce58-50   1x [CPU:0.5]     2 days ago  2m 39s         < 1s          0            SUCCEEDED  
4   test-spot-tpu-9ce1ce58-b1       1x [tpu-v2-8:1]  2 days ago  6m 53s         46s           0            SUCCEEDED  
3   test-gcp-spot-9ce1ce58-03       1x [CPU:0.5]     2 days ago  4m 17s         1m 42s        0            CANCELLED  
2   test-spot                       1x [CPU:0.5]     2 days ago  2m 42s         < 1s          0            SUCCEEDED  
1   test-down                       1x [CPU:0.5]     2 days ago  19m 27s        16m 40s       0            SUCCEEDED  
```
2. Rename `sky.spot_status` to `sky.spot_queue`, and deprecate `sky.spot_status`.
```
> python -u -c 'import sky; print(sky.spot_status(refresh=False))'                 
WARNING: `spot_status` is deprecated. Instead, use: spot_queue
[{'job_id': 9, 'job_name': 'test-log', 'resources': '1x [CPU:0.5]', 'submitted_at': 1666504336.900911, 'status': <SpotStatus.RUNNING: 'RUNNING'>, 'run_timestamp': 'sky-2022-10-23-05-52-16-900911', 'start_at': 1666504486.077749, 'end_at': None, 'last_recovered_at': 1666504486.077749, 'recovery_count': 0, 'job_duration': 4216.067179918289, 'cluster_resources': '1x AWS(m6i.2xlarge[Spot])', 'region': 'us-east-2'}, {'job_id': 8, 'job_name': 'test-logging', 'resources': '1x [CPU:0.5]', 'submitted_at': 1666465105.396012, 'status': <SpotStatus.CANCELLED: 'CANCELLED'>, 'run_timestamp': 'sky-2022-10-22-18-58-25-396012', 'start_at': 1666465272.4991996, 'end_at': 1666476011.34647, 'last_recovered_at': 1666465591.9829938, 'recovery_count': 1, 'job_duration': 10586.945924043655, 'cluster_resources': '-', 'region': '-'}, {'job_id': 7, 'job_name': 'sky-08bc-zhwu', 'resources': '1x [CPU:0.5]', 'submitted_at': 1666380755.480826, 'status': <SpotStatus.SUCCEEDED: 'SUCCEEDED'>, 'run_timestamp': 'sky-2022-10-21-19-32-35-480826', 'start_at': 1666380902.963115, 'end_at': 1666380903.9682567, 'last_recovered_at': 1666380902.963115, 'recovery_count': 0, 'job_duration': 1.0051417350769043, 'cluster_resources': '-', 'region': '-'}, {'job_id': 6, 'job_name': 'test-spot-recovery-9ce1ce58-8c', 'resources': '1x [CPU:0.5]', 'submitted_at': 1666335239.488113, 'status': <SpotStatus.CANCELLED: 'CANCELLED'>, 'run_timestamp': 'sky-2022-10-21-06-53-59-488113', 'start_at': 1666335400.7137127, 'end_at': 1666335832.7867405, 'last_recovered_at': 1666335789.0542727, 'recovery_count': 1, 'job_duration': 242.17492961883545, 'cluster_resources': '-', 'region': '-'}, {'job_id': 5, 'job_name': 'test-spot-storage-9ce1ce58-50', 'resources': '1x [CPU:0.5]', 'submitted_at': 1666335200.331433, 'status': <SpotStatus.SUCCEEDED: 'SUCCEEDED'>, 'run_timestamp': 'sky-2022-10-21-06-53-20-331433', 'start_at': 1666335359.547556, 'end_at': 1666335360.167194, 'last_recovered_at': 1666335359.547556, 'recovery_count': 0, 'job_duration': 0.6196379661560059, 'cluster_resources': '-', 'region': '-'}, {'job_id': 4, 'job_name': 'test-spot-tpu-9ce1ce58-b1', 'resources': '1x [tpu-v2-8:1]', 'submitted_at': 1666335169.900792, 'status': <SpotStatus.SUCCEEDED: 'SUCCEEDED'>, 'run_timestamp': 'sky-2022-10-21-06-52-49-900792', 'start_at': 1666335536.744013, 'end_at': 1666335583.0232415, 'last_recovered_at': 1666335536.744013, 'recovery_count': 0, 'job_duration': 46.2792284488678, 'cluster_resources': '-', 'region': '-'}, {'job_id': 3, 'job_name': 'test-gcp-spot-9ce1ce58-03', 'resources': '1x [CPU:0.5]', 'submitted_at': 1666335143.443594, 'status': <SpotStatus.CANCELLED: 'CANCELLED'>, 'run_timestamp': 'sky-2022-10-21-06-52-23-443594', 'start_at': 1666335298.0957196, 'end_at': 1666335400.6158736, 'last_recovered_at': 1666335298.0957196, 'recovery_count': 0, 'job_duration': 102.52015399932861, 'cluster_resources': '-', 'region': '-'}, {'job_id': 2, 'job_name': 'test-spot', 'resources': '1x [CPU:0.5]', 'submitted_at': 1666334074.458248, 'status': <SpotStatus.SUCCEEDED: 'SUCCEEDED'>, 'run_timestamp': 'sky-2022-10-21-06-34-34-458248', 'start_at': 1666334236.014404, 'end_at': 1666334236.6104856, 'last_recovered_at': 1666334236.014404, 'recovery_count': 0, 'job_duration': 0.5960814952850342, 'cluster_resources': '-', 'region': '-'}, {'job_id': 1, 'job_name': 'test-down', 'resources': '1x [CPU:0.5]', 'submitted_at': 1666291725.985772, 'status': <SpotStatus.SUCCEEDED: 'SUCCEEDED'>, 'run_timestamp': 'sky-2022-10-20-18-48-45-985772', 'start_at': 1666291892.5184262, 'end_at': 1666292893.3086202, 'last_recovered_at': 1666291892.5184262, 'recovery_count': 0, 'job_duration': 1000.7901940345764, 'cluster_resources': '-', 'region': '-'}]
```
3. Fix the help string after detaching from logging of `sky spot launch`.
```
I 10-23 14:17:47 cloud_vm_ray_backend.py:2014] Running setup on 1 node.
I 10-23 14:17:51 cloud_vm_ray_backend.py:2017] Setup completed.
I 10-23 14:17:55 cloud_vm_ray_backend.py:2082] Job submitted with Job ID: 12
I 10-23 21:17:56 spot_utils.py:207] Waiting for the spot controller process to be RUNNING (status: PENDING).
I 10-23 21:18:01 spot_utils.py:235] INFO: The log is not ready yet, as the spot job is STARTING. Waiting for 20 seconds.
I 10-23 21:18:21 spot_utils.py:235] INFO: The log is not ready yet, as the spot job is STARTING. Waiting for 20 seconds.
^CW 10-23 14:18:33 backend_utils.py:1994] The job will keep running after Ctrl-C.
Shared connection to 18.234.164.42 closed.
I 10-23 14:18:33 cloud_vm_ray_backend.py:2096] Spot Job ID: 12
I 10-23 14:18:33 cloud_vm_ray_backend.py:2096] To cancel the job:               sky spot cancel 12
I 10-23 14:18:33 cloud_vm_ray_backend.py:2096] To stream the logs:              sky spot logs 12
I 10-23 14:18:33 cloud_vm_ray_backend.py:2096] To stream controller logs:       sky logs sky-spot-controller-9ce1ce58 12
I 10-23 14:18:33 cloud_vm_ray_backend.py:2096] To view all spot jobs:           sky spot queue
```



Tested:
- [x] `sky spot status`
- [x] `sky spot queue`
- [x] `python -u -c 'import sky; print(sky.spot_status(refresh=False))'`
- [x] `python -u -c 'import sky; print(sky.spot_queue(refresh=False))'`        
- [x] `./tests/run_smoke_tests.sh`        

Problem statement:
[Spot] Correct hint after ctrl-c from `sky spot launch` during streaming logs
Currently, when we ctrl-c from `sky spot launch` the hint for canceling jobs should be `sky spot cancel`, but the current output is as following:
```
I 10-04 15:56:40 cloud_vm_ray_backend.py:2080] Job ID: 17
I 10-04 15:56:40 cloud_vm_ray_backend.py:2080] To cancel the job:       sky cancel sky-spot-controller-9ce1ce58 17
I 10-04 15:56:40 cloud_vm_ray_backend.py:2080] To stream the logs:      sky logs sky-spot-controller-9ce1ce58 17
I 10-04 15:56:40 cloud_vm_ray_backend.py:2080] To view the job queue:   sky queue sky-spot-controller-9ce1ce58
I 10-04 15:56:40 cloud_vm_ray_backend.py:2189] 
I 10-04 15:56:40 cloud_vm_ray_backend.py:2189] Cluster name: sky-spot-controller-9ce1ce58
I 10-04 15:56:40 cloud_vm_ray_backend.py:2189] To log into the head VM: ssh sky-spot-controller-9ce1ce58
I 10-04 15:56:40 cloud_vm_ray_backend.py:2189] To submit a job:         sky exec sky-spot-controller-9ce1ce58 yaml_file
I 10-04 15:56:40 cloud_vm_ray_backend.py:2189] To stop the cluster:     sky stop sky-spot-controller-9ce1ce58
I 10-04 15:56:40 cloud_vm_ray_backend.py:2189] To teardown the cluster: sky down sky-spot-controller-9ce1ce58
```

UX: spot and non-spot versions use different names to see existing jobs: `sky queue` vs `sky spot status`
A comment from a new user:

 I found it strange that the spot and non-spot versions use different names for similar tasks.
* to see existing jobs: `sky queue` vs `sky spot status`

-----

We can consider renaming options.

Hints from issue discussion:
What is the final decision regarding this? I saw that `sky spot queue` and `sky queue` is the best option.
Yes, according to the vote we should rename `spot status` to `spot queue`
everywhere.

On Sat, Oct 22, 2022 at 7:58 PM Michael Luo ***@***.***>
wrote:

> What is the final decision regarding this? I saw that sky spot queue and sky
> queue is the best option.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/skypilot-org/skypilot/issues/1277#issuecomment-1287989956>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAEQWHTB4QKQHHZQR6Q27NTWESSVHANCNFSM6AAAAAARJSK3SE>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>

Commit message: fix

Patch to review:
diff --git a/docs/source/examples/spot-jobs.rst b/docs/source/examples/spot-jobs.rst
index 04c0b270ab0..d091a46b070 100644
--- a/docs/source/examples/spot-jobs.rst
+++ b/docs/source/examples/spot-jobs.rst
@@ -188,7 +188,7 @@ Here are some commands for managed spot jobs. Check :code:`sky spot --help` for
 .. code-block:: console
 
     # Check the status of the spot jobs
-    $ sky spot status
+    $ sky spot queue
     Fetching managed spot job status...
     Managed spot jobs:
     ID NAME     RESOURCES     SUBMITTED   TOT. DURATION   JOB DURATION   #RECOVERIES  STATUS
diff --git a/docs/source/reference/cli.rst b/docs/source/reference/cli.rst
index 53da6ae5c47..94007321797 100644
--- a/docs/source/reference/cli.rst
+++ b/docs/source/reference/cli.rst
@@ -86,8 +86,8 @@ Managed Spot Jobs CLI
    :prog: sky spot launch
    :nested: full
 
-.. click:: sky.cli:spot_status
-   :prog: sky spot status
+.. click:: sky.cli:spot_queue
+   :prog: sky spot queue
    :nested: full
 
 .. click:: sky.cli:spot_cancel
diff --git a/sky/__init__.py b/sky/__init__.py
index 2f6d95e0d77..db912af150e 100644
--- a/sky/__init__.py
+++ b/sky/__init__.py
@@ -20,8 +20,8 @@
 from sky.global_user_state import ClusterStatus
 from sky.skylet.job_lib import JobStatus
 from sky.core import (status, start, stop, down, autostop, queue, cancel,
-                      tail_logs, download_logs, job_status, spot_status,
-                      spot_cancel, storage_ls, storage_delete)
+                      tail_logs, download_logs, job_status, spot_queue,
+                      spot_status, spot_cancel, storage_ls, storage_delete)
 
 # Aliases.
 AWS = clouds.AWS
@@ -68,7 +68,8 @@
     'download_logs',
     'job_status',
     # core APIs Spot Job Management
-    'spot_status',
+    'spot_queue',
+    'spot_status',  # Deprecated (alias for spot_queue)
     'spot_cancel',
     # core APIs Storage Management
     'storage_ls',
diff --git a/sky/backends/backend.py b/sky/backends/backend.py
index 272ffb7e3e1..dd6e522fc4f 100644
--- a/sky/backends/backend.py
+++ b/sky/backends/backend.py
@@ -82,9 +82,9 @@ def execute(self, handle: ResourceHandle, task: 'task_lib.Task',
         return self._execute(handle, task, detach_run)
 
     @timeline.event
-    def post_execute(self, handle: ResourceHandle, teardown: bool) -> None:
+    def post_execute(self, handle: ResourceHandle) -> None:
         """Post execute(): e.g., print helpful inspection messages."""
-        return self._post_execute(handle, teardown)
+        return self._post_execute(handle)
 
     @timeline.event
     def teardown_ephemeral_storage(self, task: 'task_lib.Task') -> None:
@@ -130,7 +130,7 @@ def _execute(self, handle: ResourceHandle, task: 'task_lib.Task',
                  detach_run: bool) -> None:
         raise NotImplementedError
 
-    def _post_execute(self, handle: ResourceHandle, teardown: bool) -> None:
+    def _post_execute(self, handle: ResourceHandle) -> None:
         raise NotImplementedError
 
     def _teardown_ephemeral_storage(self, task: 'task_lib.Task') -> None:
diff --git a/sky/backends/cloud_vm_ray_backend.py b/sky/backends/cloud_vm_ray_backend.py
index fc22bdf91a6..e7fd8f481e2 100644
--- a/sky/backends/cloud_vm_ray_backend.py
+++ b/sky/backends/cloud_vm_ray_backend.py
@@ -198,7 +198,7 @@ def add_prologue(self,
             'run_bash_command_with_log = ray.remote(run_bash_command_with_log)',
         ]
         if spot_task is not None:
-            # Add the spot job to spot status table.
+            # Add the spot job to spot queue table.
             resources_str = backend_utils.get_task_resources_str(spot_task)
             self._code += [
                 'from sky.spot import spot_state',
@@ -2092,17 +2092,33 @@ def _exec_code_on_head(
                     self.tail_logs(handle, job_id)
         finally:
             name = handle.cluster_name
-            logger.info(f'{fore.CYAN}Job ID: '
-                        f'{style.BRIGHT}{job_id}{style.RESET_ALL}'
-                        '\nTo cancel the job:\t'
-                        f'{backend_utils.BOLD}sky cancel {name} {job_id}'
-                        f'{backend_utils.RESET_BOLD}'
-                        '\nTo stream the logs:\t'
-                        f'{backend_utils.BOLD}sky logs {name} {job_id}'
-                        f'{backend_utils.RESET_BOLD}'
-                        '\nTo view the job queue:\t'
-                        f'{backend_utils.BOLD}sky queue {name}'
-                        f'{backend_utils.RESET_BOLD}')
+            if name == spot_lib.SPOT_CONTROLLER_NAME:
+                logger.info(f'{fore.CYAN}Spot Job ID: '
+                            f'{style.BRIGHT}{job_id}{style.RESET_ALL}'
+                            '\nTo cancel the job:\t'
+                            f'{backend_utils.BOLD}sky spot cancel {job_id}'
+                            f'{backend_utils.RESET_BOLD}'
+                            '\nTo stream the logs:\t'
+                            f'{backend_utils.BOLD}sky spot logs {job_id}'
+                            f'{backend_utils.RESET_BOLD}'
+                            f'\nTo stream debug logs:\t'
+                            f'{backend_utils.BOLD}sky logs {name} {job_id}'
+                            f'{backend_utils.RESET_BOLD}'
+                            '\nTo view all spot jobs:\t'
+                            f'{backend_utils.BOLD}sky spot queue'
+                            f'{backend_utils.RESET_BOLD}')
+            else:
+                logger.info(f'{fore.CYAN}Job ID: '
+                            f'{style.BRIGHT}{job_id}{style.RESET_ALL}'
+                            '\nTo cancel the job:\t'
+                            f'{backend_utils.BOLD}sky cancel {name} {job_id}'
+                            f'{backend_utils.RESET_BOLD}'
+                            '\nTo stream the logs:\t'
+                            f'{backend_utils.BOLD}sky logs {name} {job_id}'
+                            f'{backend_utils.RESET_BOLD}'
+                            '\nTo view the job queue:\t'
+                            f'{backend_utils.BOLD}sky queue {name}'
+                            f'{backend_utils.RESET_BOLD}')
 
     def _setup_and_create_job_cmd_on_local_head(
         self,
@@ -2190,31 +2206,32 @@ def _execute(
             # Case: task_lib.Task(run, num_nodes=1)
             self._execute_task_one_node(handle, task, job_id, detach_run)
 
-    def _post_execute(self, handle: ResourceHandle, teardown: bool) -> None:
+    def _post_execute(self, handle: ResourceHandle) -> None:
         colorama.init()
         fore = colorama.Fore
         style = colorama.Style
         name = handle.cluster_name
+        if name == spot_lib.SPOT_CONTROLLER_NAME:
+            return
         stop_str = ('\nTo stop the cluster:'
                     f'\t{backend_utils.BOLD}sky stop {name}'
                     f'{backend_utils.RESET_BOLD}')
         if isinstance(handle.launched_resources.cloud, clouds.Local):
             stop_str = ''
-        if not teardown:
-            logger.info(f'\n{fore.CYAN}Cluster name: '
-                        f'{style.BRIGHT}{name}{style.RESET_ALL}'
-                        '\nTo log into the head VM:\t'
-                        f'{backend_utils.BOLD}ssh {name}'
-                        f'{backend_utils.RESET_BOLD}'
-                        '\nTo submit a job:'
-                        f'\t\t{backend_utils.BOLD}sky exec {name} yaml_file'
-                        f'{backend_utils.RESET_BOLD}'
-                        f'{stop_str}'
-                        '\nTo teardown the cluster:'
-                        f'\t{backend_utils.BOLD}sky down {name}'
-                        f'{backend_utils.RESET_BOLD}')
-            if handle.tpu_delete_script is not None:
-                logger.info('Tip: `sky down` will delete launched TPU(s) too.')
+        logger.info(f'\n{fore.CYAN}Cluster name: '
+                    f'{style.BRIGHT}{name}{style.RESET_ALL}'
+                    '\nTo log into the head VM:\t'
+                    f'{backend_utils.BOLD}ssh {name}'
+                    f'{backend_utils.RESET_BOLD}'
+                    '\nTo submit a job:'
+                    f'\t\t{backend_utils.BOLD}sky exec {name} yaml_file'
+                    f'{backend_utils.RESET_BOLD}'
+                    f'{stop_str}'
+                    '\nTo teardown the cluster:'
+                    f'\t{backend_utils.BOLD}sky down {name}'
+                    f'{backend_utils.RESET_BOLD}')
+        if handle.tpu_delete_script is not None:
+            logger.info('Tip: `sky down` will delete launched TPU(s) too.')
 
     def _teardown_ephemeral_storage(self, task: task_lib.Task) -> None:
         storage_mounts = task.storage_mounts
diff --git a/sky/backends/local_docker_backend.py b/sky/backends/local_docker_backend.py
index cb00582830f..5133a85081a 100644
--- a/sky/backends/local_docker_backend.py
+++ b/sky/backends/local_docker_backend.py
@@ -268,7 +268,7 @@ def _execute(self, handle: ResourceHandle, task: 'task_lib.Task',
 
         self._execute_task_one_node(handle, task)
 
-    def _post_execute(self, handle: ResourceHandle, teardown: bool) -> None:
+    def _post_execute(self, handle: ResourceHandle) -> None:
         colorama.init()
         style = colorama.Style
         container = self.containers[handle]
diff --git a/sky/cli.py b/sky/cli.py
index 365adecd018..f882699718e 100644
--- a/sky/cli.py
+++ b/sky/cli.py
@@ -27,6 +27,7 @@
 listed in "sky --help".  Take care to put logically connected commands close to
 each other.
 """
+import copy
 import datetime
 import functools
 import getpass
@@ -895,32 +896,52 @@ def _make_dag_from_entrypoint_with_overrides(
     return dag
 
 
-def _start_cluster(cluster_name: str,
-                   idle_minutes_to_autostop: Optional[int] = None,
-                   retry_until_up: bool = False):
-    handle = global_user_state.get_handle_from_cluster_name(cluster_name)
-    backend = backend_utils.get_backend_from_handle(handle)
-    assert isinstance(backend, backends.CloudVmRayBackend)
-    with sky.Dag():
-        dummy_task = sky.Task().set_resources(handle.launched_resources)
-        dummy_task.num_nodes = handle.launched_nodes
-    handle = backend.provision(dummy_task,
-                               to_provision=handle.launched_resources,
-                               dryrun=False,
-                               stream_logs=True,
-                               cluster_name=cluster_name,
-                               retry_until_up=retry_until_up)
-    if idle_minutes_to_autostop is not None:
-        backend.set_autostop(handle, idle_minutes_to_autostop)
-    return handle
-
-
-class _NaturalOrderGroup(click.Group):
-    """Lists commands in the order they are defined in this script.
+class _NaturalOrderWithAliasesGroup(click.Group):
+    """Lists commands in the order defined in this script and support aliases.
 
     Reference: https://github.com/pallets/click/issues/513
     """
 
+    def command(self, *args, **kwargs):
+        """Support `aliases` as a list in kwargs."""
+
+        aliases = kwargs.pop('aliases', [])
+        super_cls = super()
+        if not aliases:
+            return super_cls.command(*args, **kwargs)
+
+        def _decorator(f):
+            params = None
+            if hasattr(f, '__click_params__'):
+                params = copy.copy(f.__click_params__)
+            cmd = super_cls.command(*args, **kwargs)(f)
+
+            kwargs.pop('name', None)
+            new_args = args[1:] if args else []
+
+            f.__doc__ = f'[Deprecated] Alias for `{cmd.name}`.'
+
+            def deprecate(f, alias):
+
+                @functools.wraps(f)
+                def wrapper(*args, **kwargs):
+                    click.secho(
+                        f'WARNING: `{self.name} {alias}` is deprecated. '
+                        f'Instead, use: {self.name} {cmd.name}\n',
+                        fg='yellow')
+                    return f(*args, **kwargs)
+
+                return wrapper
+
+            for alias in aliases:
+                if params is not None:
+                    f.__click_params__ = copy.copy(params)
+                super_cls.command(alias, *new_args,
+                                  **kwargs)(deprecate(f, alias))
+            return cmd
+
+        return _decorator
+
     def list_commands(self, ctx):
         return self.commands.keys()
 
@@ -940,7 +961,8 @@ def get_help(self, ctx):
         return super().get_help(ctx)
 
 
-@click.group(cls=_NaturalOrderGroup, context_settings=_CONTEXT_SETTINGS)
+@click.group(cls=_NaturalOrderWithAliasesGroup,
+             context_settings=_CONTEXT_SETTINGS)
 @click.option('--install-shell-completion',
               type=click.Choice(['bash', 'zsh', 'fish', 'auto']),
               callback=_install_shell_completion,
@@ -1890,10 +1912,10 @@ def _hint_for_down_spot_controller(controller_name: str):
            f'spot controller ({cluster_status.value}). Please be '
            f'aware of the following:{colorama.Style.RESET_ALL}'
            '\n * All logs and status information of the spot '
-           'jobs (output of `sky spot status`) will be lost.')
+           'jobs (output of `sky spot queue`) will be lost.')
     if cluster_status == global_user_state.ClusterStatus.UP:
         try:
-            spot_jobs = core.spot_status(refresh=False)
+            spot_jobs = core.spot_queue(refresh=False)
         except exceptions.ClusterNotUpError:
             # The spot controller cluster status changed during querying
             # the spot jobs, use the latest cluster status, so that the
@@ -2454,7 +2476,7 @@ def _output():
         click.echo()
 
 
-@cli.group(cls=_NaturalOrderGroup)
+@cli.group(cls=_NaturalOrderWithAliasesGroup)
 def storage():
     """Storage related commands."""
     pass
@@ -2511,7 +2533,7 @@ def storage_delete(names: Tuple[str], all: bool):  # pylint: disable=redefined-b
         sky.storage_delete(name)
 
 
-@cli.group(cls=_NaturalOrderGroup)
+@cli.group(cls=_NaturalOrderWithAliasesGroup)
 def admin():
     """Sky administrator commands for local clusters."""
     pass
@@ -2607,7 +2629,7 @@ def _is_spot_controller_up(
     return controller_status, handle
 
 
-@cli.group(cls=_NaturalOrderGroup)
+@cli.group(cls=_NaturalOrderWithAliasesGroup)
 def spot():
     """Managed spot instances related commands."""
     pass
@@ -2707,7 +2729,7 @@ def spot_launch(
                     retry_until_up=retry_until_up)
 
 
-@spot.command('status', cls=_DocumentedCodeCommand)
+@spot.command('queue', aliases=['status'], cls=_DocumentedCodeCommand)
 @click.option('--all',
               '-a',
               default=False,
@@ -2722,10 +2744,10 @@ def spot_launch(
     required=False,
     help='Query the latest statuses, restarting the spot controller if stopped.'
 )
-@usage_lib.entrypoint
+# @usage_lib.entrypoint
 # pylint: disable=redefined-builtin
-def spot_status(all: bool, refresh: bool):
-    """Show statuses of managed spot jobs.
+def spot_queue(all: bool, refresh: bool):
+    """Show managed spot job queue.
 
     \b
     Each spot job can have one of the following statuses:
@@ -2745,17 +2767,17 @@ def spot_status(all: bool, refresh: bool):
 
     If the job failed, either due to user code or spot unavailability, the error
     log can be found with ``sky logs sky-spot-controller-<user_hash> job_id``.
-    Please find your exact spot controller name with ``sky status -a``.
+    Please find your exact spot controller name with ``sky status``.
 
     (Tip) To fetch job statuses every 60 seconds, use ``watch``:
 
     .. code-block:: bash
 
-      watch -n60 sky spot status
+      watch -n60 sky spot queue
     """
     click.secho('Fetching managed spot job statuses...', fg='yellow')
     try:
-        job_table = core.spot_status(refresh=refresh)
+        job_table = core.spot_queue(refresh=refresh)
     except exceptions.ClusterNotUpError:
         cache = spot_lib.load_job_table_cache()
         if cache is not None:
@@ -2902,7 +2924,7 @@ def _get_candidate_configs(yaml_path: str) -> Optional[List[Dict[str, str]]]:
     return candidates
 
 
-@cli.group(cls=_NaturalOrderGroup)
+@cli.group(cls=_NaturalOrderWithAliasesGroup)
 def bench():
     """Sky Benchmark related commands."""
     pass
diff --git a/sky/core.py b/sky/core.py
index 414c3e7db41..5c7ad7a2189 100644
--- a/sky/core.py
+++ b/sky/core.py
@@ -1,6 +1,7 @@
 """SDK functions for cluster/job management."""
 import colorama
 import getpass
+import sys
 from typing import Any, Dict, List, Optional, Tuple
 
 from sky import dag
@@ -81,7 +82,7 @@ def _start(
             f'Starting cluster {cluster_name!r} with backend {backend.NAME} '
             'is not supported.')
 
-    # NOTE: if spot_status() calls _start() and hits here, that entrypoint
+    # NOTE: if spot_queue() calls _start() and hits here, that entrypoint
     # would have a cluster name (the controller) filled in.
     usage_lib.record_cluster_name_for_current_operation(cluster_name)
 
@@ -477,9 +478,18 @@ def _is_spot_controller_up(
 
 @usage_lib.entrypoint
 def spot_status(refresh: bool) -> List[Dict[str, Any]]:
+    print(
+        f'{colorama.Fore.YELLOW}WARNING: `spot_status` is deprecated. '
+        f'Instead, use: spot_queue{colorama.Style.RESET_ALL}',
+        file=sys.stderr)
+    return spot_queue(refresh=refresh)
+
+
+@usage_lib.entrypoint
+def spot_queue(refresh: bool) -> List[Dict[str, Any]]:
     """Get statuses of managed spot jobs.
 
-    Please refer to the sky.cli.spot_status for the document.
+    Please refer to the sky.cli.spot_queue for the document.
 
     Returns:
         [
@@ -502,7 +512,7 @@ def spot_status(refresh: bool) -> List[Dict[str, Any]]:
 
     stop_msg = ''
     if not refresh:
-        stop_msg = 'To view the latest job table: sky spot status --refresh'
+        stop_msg = 'To view the latest job table: sky spot queue --refresh'
     controller_status, handle = _is_spot_controller_up(stop_msg)
 
     if controller_status is None:
diff --git a/sky/execution.py b/sky/execution.py
index 95db48a98b5..e9a63e76e86 100644
--- a/sky/execution.py
+++ b/sky/execution.py
@@ -246,7 +246,7 @@ def _execute(
                 backend.execute(handle, task, detach_run)
             finally:
                 # Enables post_execute() to be run after KeyboardInterrupt.
-                backend.post_execute(handle, down)
+                backend.post_execute(handle)
 
         if Stage.DOWN in stages:
             if down and idle_minutes_to_autostop is None:
diff --git a/sky/spot/spot_utils.py b/sky/spot/spot_utils.py
index 1784d986a7b..ad3738f5060 100644
--- a/sky/spot/spot_utils.py
+++ b/sky/spot/spot_utils.py
@@ -229,14 +229,14 @@ def stream_logs_by_id(job_id: int, follow: bool = True) -> str:
     task_name = spot_state.get_task_name_by_job_id(job_id)
     cluster_name = generate_spot_cluster_name(task_name, job_id)
     backend = backends.CloudVmRayBackend()
-    spot_status = spot_state.get_status(job_id)
-    while not spot_status.is_terminal():
-        if spot_status != spot_state.SpotStatus.RUNNING:
+    spot_queue = spot_state.get_status(job_id)
+    while not spot_queue.is_terminal():
+        if spot_queue != spot_state.SpotStatus.RUNNING:
             logger.info(f'INFO: The log is not ready yet, as the spot job '
-                        f'is {spot_status.value}. '
+                        f'is {spot_queue.value}. '
                         f'Waiting for {JOB_STATUS_CHECK_GAP_SECONDS} seconds.')
             time.sleep(JOB_STATUS_CHECK_GAP_SECONDS)
-            spot_status = spot_state.get_status(job_id)
+            spot_queue = spot_state.get_status(job_id)
             continue
         handle = global_user_state.get_handle_from_cluster_name(cluster_name)
         returncode = backend.tail_logs(handle,
@@ -259,11 +259,11 @@ def stream_logs_by_id(job_id: int, follow: bool = True) -> str:
             f'in {JOB_STATUS_CHECK_GAP_SECONDS} seconds.')
         # If the tailing fails, it is likely that the cluster fails, so we wait
         # a while to make sure the spot state is updated by the controller, and
-        # check the spot status again.
+        # check the spot queue again.
         time.sleep(JOB_STATUS_CHECK_GAP_SECONDS)
-        spot_status = spot_state.get_status(job_id)
+        spot_queue = spot_state.get_status(job_id)
     else:
-        # The spot_status is in terminal state.
+        # The spot_queue is in terminal state.
         logger.info(f'Logs finished for job {job_id} '
                     f'(status: {spot_state.get_status(job_id).value}).')
     return ''
diff --git a/tests/test_smoke.py b/tests/test_smoke.py
index 76162a7cd44..51a0861c802 100644
--- a/tests/test_smoke.py
+++ b/tests/test_smoke.py
@@ -659,12 +659,12 @@ def test_spot():
             f'sky spot launch -n {name}-1 examples/managed_spot.yaml -y -d',
             f'sky spot launch -n {name}-2 examples/managed_spot.yaml -y -d',
             'sleep 5',
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name}-1 | head -n1 | grep "STARTING\|RUNNING"',
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name}-2 | head -n1 | grep "STARTING\|RUNNING"',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name}-1 | head -n1 | grep "STARTING\|RUNNING"',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name}-2 | head -n1 | grep "STARTING\|RUNNING"',
             f'sky spot cancel -y -n {name}-1',
             'sleep 200',
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name}-1 | head -n1 | grep CANCELLED',
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name}-2 | head -n1 | grep "RUNNING\|SUCCEEDED"',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name}-1 | head -n1 | grep CANCELLED',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name}-2 | head -n1 | grep "RUNNING\|SUCCEEDED"',
         ],
         f'sky spot cancel -y -n {name}-1; sky spot cancel -y -n {name}-2',
     )
@@ -682,9 +682,9 @@ def test_gcp_spot():
             'sleep 5',
             # Captures & prints the table for easier debugging. Two echo's to
             # separate the table from the grep output.
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep STARTING',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep STARTING',
             'sleep 200',
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep RUNNING',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep RUNNING',
         ],
         f'sky spot cancel -y -n {name}',
     )
@@ -701,7 +701,7 @@ def test_spot_recovery():
         [
             f'sky spot launch --cloud aws --region {region} -n {name} "echo SKYPILOT_RUN_ID: $SKYPILOT_RUN_ID; sleep 1000"  -y -d',
             'sleep 300',
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep "RUNNING"',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep "RUNNING"',
             f'RUN_ID=$(sky spot logs -n {name} --no-follow | grep SKYPILOT_RUN_ID | cut -d: -f2); echo $RUN_ID',
             # Terminate the cluster manually.
             (f'aws ec2 terminate-instances --region {region} --instance-ids $('
@@ -710,9 +710,9 @@ def test_spot_recovery():
              f'--query Reservations[].Instances[].InstanceId '
              '--output text)'),
             'sleep 50',
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep "RECOVERING"',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep "RECOVERING"',
             'sleep 200',
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep "RUNNING"',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep "RUNNING"',
             f'sky spot logs -n {name} --no-follow | grep SKYPILOT_RUN_ID | grep "$RUN_ID"',
         ],
         f'sky spot cancel -y -n {name}',
@@ -736,8 +736,8 @@ def test_spot_storage():
             'managed-spot-storage',
             [
                 f'sky spot launch -n {name} {file_path} -y',
-                'sleep 60',  # Wait the spot status to be updated
-                f'sky spot status | grep {name} | grep SUCCEEDED',
+                'sleep 60',  # Wait the spot queue to be updated
+                f'sky spot queue | grep {name} | grep SUCCEEDED',
                 f'[ $(aws s3api list-buckets --query "Buckets[?contains(Name, \'{storage_name}\')].Name" --output text | wc -l) -eq 0 ]'
             ],
             f'sky spot cancel -y -n {name}',
@@ -754,9 +754,9 @@ def test_spot_tpu():
         [
             f'sky spot launch -n {name} examples/tpu/tpuvm_mnist.yaml -y -d',
             'sleep 5',
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep STARTING',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep STARTING',
             'sleep 600',  # TPU takes a while to launch
-            f's=$(sky spot status); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep "RUNNING\|SUCCEEDED"',
+            f's=$(sky spot queue); printf "$s"; echo; echo; printf "$s" | grep {name} | head -n1 | grep "RUNNING\|SUCCEEDED"',
         ],
         f'sky spot cancel -y -n {name}',
     )
@@ -789,7 +789,7 @@ def test_inline_spot_env():
         [
             f'sky spot launch -n {name} -y --env TEST_ENV="hello world" -- "([[ ! -z \\"\$TEST_ENV\\" ]] && [[ ! -z \\"\$SKY_NODE_IPS\\" ]] && [[ ! -z \\"\$SKY_NODE_RANK\\" ]]) || exit 1"',
             'sleep 10',
-            f's=$(sky spot status) && printf "$s" && echo "$s"  | grep {name} | grep SUCCEEDED',
+            f's=$(sky spot queue) && printf "$s" && echo "$s"  | grep {name} | grep SUCCEEDED',
         ],
         f'sky spot cancel -y -n {name}',
     )
diff --git a/tests/test_spot.py b/tests/test_spot.py
index f80f47ea866..30f0e0e92a0 100644
--- a/tests/test_spot.py
+++ b/tests/test_spot.py
@@ -109,7 +109,7 @@ def mock_cluster_refresh_up(
             'sky.backends.backend_utils.refresh_cluster_status_handle',
             mock_cluster_refresh_up)
 
-        monkeypatch.setattr('sky.core.spot_status', lambda refresh: [])
+        monkeypatch.setattr('sky.core.spot_queue', lambda refresh: [])
 
         cli_runner = cli_testing.CliRunner()
         result = cli_runner.invoke(cli.down, [spot.SPOT_CONTROLLER_NAME],


Code review comments:
diff_hunk: @@ -2190,31 +2206,32 @@ def _execute(
             # Case: task_lib.Task(run, num_nodes=1)
             self._execute_task_one_node(handle, task, job_id, detach_run)
 
-    def _post_execute(self, handle: ResourceHandle, teardown: bool) -> None:
+    def _post_execute(self, handle: ResourceHandle) -> None:
path: sky/backends/cloud_vm_ray_backend.py
text: @user2:
I think we should keep the `teardown` arg, for reason below:

1. I ran `sky launch --down -c myclus minimal.yaml` and alt-tabbed to something else
2. Came back to my terminal, saw messages:
```
I 10-23 19:28:56 cloud_vm_ray_backend.py:2111] Job ID: 1
I 10-23 19:28:56 cloud_vm_ray_backend.py:2111] To cancel the job:       sky cancel myclus 1
I 10-23 19:28:56 cloud_vm_ray_backend.py:2111] To stream the logs:      sky logs myclus 1
I 10-23 19:28:56 cloud_vm_ray_backend.py:2111] To view the job queue:   sky queue myclus
I 10-23 19:28:56 cloud_vm_ray_backend.py:2221]
I 10-23 19:28:56 cloud_vm_ray_backend.py:2221] Cluster name: myclus
I 10-23 19:28:56 cloud_vm_ray_backend.py:2221] To log into the head VM: ssh myclus
I 10-23 19:28:56 cloud_vm_ray_backend.py:2221] To submit a job:         sky exec myclus yaml_file
I 10-23 19:28:56 cloud_vm_ray_backend.py:2221] To stop the cluster:     sky stop myclus
I 10-23 19:28:56 cloud_vm_ray_backend.py:2221] To teardown the cluster: sky down myclus
```
3. Tried the suggested commands, they didn't work:
```
(base) romilb@user1:/mnt/d/Romil/Berkeley/Research/sky-experiments/examples$ ssh myclus
ssh: connect to host 18.232.157.190 port 22: Connection refused
(base) romilb@user1:/mnt/d/Romil/Berkeley/Research/sky-experiments/examples$ sky cancel myclus 1
sky.exceptions.ClusterNotUpError: Cluster 'myclus' does not exist. skipped.
```

Since we set an auto-down of 1 minute, for `--down`, which is too short for the above commands to be useful, maybe we should not print them.

@author:
Good point! I disable the logging when `down` is specified for now.

diff_hunk: @@ -895,32 +896,52 @@ def _make_dag_from_entrypoint_with_overrides(
     return dag
 
 
-def _start_cluster(cluster_name: str,
-                   idle_minutes_to_autostop: Optional[int] = None,
-                   retry_until_up: bool = False):
-    handle = global_user_state.get_handle_from_cluster_name(cluster_name)
-    backend = backend_utils.get_backend_from_handle(handle)
-    assert isinstance(backend, backends.CloudVmRayBackend)
-    with sky.Dag():
-        dummy_task = sky.Task().set_resources(handle.launched_resources)
-        dummy_task.num_nodes = handle.launched_nodes
-    handle = backend.provision(dummy_task,
-                               to_provision=handle.launched_resources,
-                               dryrun=False,
-                               stream_logs=True,
-                               cluster_name=cluster_name,
-                               retry_until_up=retry_until_up)
-    if idle_minutes_to_autostop is not None:
-        backend.set_autostop(handle, idle_minutes_to_autostop)
-    return handle
-
-
-class _NaturalOrderGroup(click.Group):
-    """Lists commands in the order they are defined in this script.
+class _NaturalOrderWithAliasesGroup(click.Group):
path: sky/cli.py
text: @user1:
I feel this may be a bit too complicated.

How about we simply don't show `spot status` in `sky -h`, but still (1) make it available and alias the new `spot queue` (2) print the warning?

Example: https://github.com/ray-project/ray/blob/11e408908e1a1cac12f0699c5f970fcabff84801/python/ray/scripts/scripts.py#L2551

@author:
Great finding! Changed to the suggested implementation.

diff_hunk: @@ -188,7 +188,7 @@ Here are some commands for managed spot jobs. Check :code:`sky spot --help` for
 .. code-block:: console
 
     # Check the status of the spot jobs
-    $ sky spot status
+    $ sky spot queue
     Fetching managed spot job status...
path: docs/source/examples/spot-jobs.rst
text: @user1:
nit: actual output `status` -> `statuses`

@author:
Good catch! Fixed. Thanks!

diff_hunk: @@ -2722,10 +2744,10 @@ def spot_launch(
     required=False,
     help='Query the latest statuses, restarting the spot controller if stopped.'
 )
-@usage_lib.entrypoint
+# @usage_lib.entrypoint
path: sky/cli.py
text: @user1:
remnant?

@author:
Good catch! Fixed. Thanks!

diff_hunk: @@ -477,9 +478,18 @@ def _is_spot_controller_up(
 
 @usage_lib.entrypoint
 def spot_status(refresh: bool) -> List[Dict[str, Any]]:
+    print(
+        f'{colorama.Fore.YELLOW}WARNING: `spot_status` is deprecated. '
+        f'Instead, use: spot_queue{colorama.Style.RESET_ALL}',
+        file=sys.stderr)
+    return spot_queue(refresh=refresh)
+
+
+@usage_lib.entrypoint
+def spot_queue(refresh: bool) -> List[Dict[str, Any]]:
     """Get statuses of managed spot jobs.
 
-    Please refer to the sky.cli.spot_status for the document.
+    Please refer to the sky.cli.spot_queue for the document.
path: sky/core.py
text: @user1:
nit: *documentation

@author:
Fixed. Thanks!