Repository: ansible/awx

Issue:
Job failed if Job Slicing and Fact Storage enabled together
### Please confirm the following

- [X] I agree to follow this project's [code of conduct](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html).
- [X] I have checked the [current issues](https://github.com/ansible/awx/issues) for duplicates.
- [X] I understand that AWX is open source software provided for free and that I might not receive a timely response.

### Bug Summary

Job failed if Job Slicing and Fact Storage enabled together

`Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/tasks/jobs.py", line 504, in run
    self.instance.start_job_fact_cache(
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/models/jobs.py", line 862, in start_job_fact_cache
    hosts = hosts.filter(ansible_facts_modified__gte=timeout)
AttributeError: 'list' object has no attribute 'filter'`

### AWX version

AWX 21.7.0  

### Select the relevant components

- [ ] UI
- [ ] API
- [ ] Docs
- [ ] Collection
- [ ] CLI
- [ ] Other

### Installation method

kubernetes

### Modifications

no

### Ansible version

_No response_

### Operating system

_No response_

### Web browser

_No response_

### Steps to reproduce

1. Create Job Template. 
2. Make sure the Job Slicing >2.
3. Enable Enable Fact Storage flag
4. Launch job 


### Expected results

Job is done

### Actual results

Job failed with error:
`Traceback (most recent call last):
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/tasks/jobs.py", line 504, in run
    self.instance.start_job_fact_cache(
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/awx/main/models/jobs.py", line 862, in start_job_fact_cache
    hosts = hosts.filter(ansible_facts_modified__gte=timeout)
AttributeError: 'list' object has no attribute 'filter'`

### Additional information

_No response_


Discussion:
I confirmed I can reproduce this error. Doing so requires modifying the setting `ANSIBLE_FACT_CACHE_TIMEOUT` to a value greater than zero, @relrod we may need to evaluate the extent of the fallout if this was introduced in https://github.com/ansible/awx/pull/12790 I think the root of the problem is that a slice will return a list, not a queryset.

I also confirmed that this is fixed in https://github.com/ansible/awx/pull/12998, so I don't want to dive into a separate AWX patch just for this bug.
> I think the root of the problem is that a slice will return a list, not a queryset.

Yep. Per Django docs:

>  Slicing an unevaluated `QuerySet` usually returns another unevaluated `QuerySet`, but Django will execute the database query if you use the “step” parameter of slice syntax, and will return a list.

I did not realize that was a special case (but it makes sense since it probably can't do the step part as part of the query). Ugh. :(
Also, thanks for this report @lutskevich

Resolving patch:
diff --git a/awx/main/models/inventory.py b/awx/main/models/inventory.py
--- a/awx/main/models/inventory.py
+++ b/awx/main/models/inventory.py
@@ -247,6 +247,19 @@ def parse_slice_params(slice_str):
         return (number, step)
 
     def get_sliced_hosts(self, host_queryset, slice_number, slice_count):
+        """
+        Returns a slice of Hosts given a slice number and total slice count, or
+        the original queryset if slicing is not requested.
+
+        NOTE: If slicing is performed, this will return a List[Host] with the
+        resulting slice. If slicing is not performed it will return the
+        original queryset (not evaluating it or forcing it to a list). This
+        puts the burden on the caller to check the resulting type. This is
+        non-ideal because it's easy to get wrong, but I think the only way
+        around it is to force the queryset which has memory implications for
+        large inventories.
+        """
+
         if slice_count > 1 and slice_number > 0:
             offset = slice_number - 1
             host_queryset = host_queryset[offset::slice_count]
diff --git a/awx/main/models/jobs.py b/awx/main/models/jobs.py
--- a/awx/main/models/jobs.py
+++ b/awx/main/models/jobs.py
@@ -15,6 +15,7 @@
 from django.conf import settings
 from django.core.exceptions import ValidationError
 from django.db import models
+from django.db.models.query import QuerySet
 
 # from django.core.cache import cache
 from django.utils.encoding import smart_str
@@ -844,22 +845,30 @@ def get_notification_templates(self):
     def get_notification_friendly_name(self):
         return "Job"
 
-    def _get_inventory_hosts(self, only=['name', 'ansible_facts', 'ansible_facts_modified', 'modified', 'inventory_id']):
+    def _get_inventory_hosts(self, only=('name', 'ansible_facts', 'ansible_facts_modified', 'modified', 'inventory_id'), **filters):
+        """Return value is an iterable for the relevant hosts for this job"""
         if not self.inventory:
             return []
         host_queryset = self.inventory.hosts.only(*only)
-        return self.inventory.get_sliced_hosts(host_queryset, self.job_slice_number, self.job_slice_count)
+        if filters:
+            host_queryset = host_queryset.filter(**filters)
+        host_queryset = self.inventory.get_sliced_hosts(host_queryset, self.job_slice_number, self.job_slice_count)
+        if isinstance(host_queryset, QuerySet):
+            return host_queryset.iterator()
+        return host_queryset
 
     def start_job_fact_cache(self, destination, modification_times, timeout=None):
         self.log_lifecycle("start_job_fact_cache")
         os.makedirs(destination, mode=0o700)
-        hosts = self._get_inventory_hosts()
+
         if timeout is None:
             timeout = settings.ANSIBLE_FACT_CACHE_TIMEOUT
         if timeout > 0:
             # exclude hosts with fact data older than `settings.ANSIBLE_FACT_CACHE_TIMEOUT seconds`
             timeout = now() - datetime.timedelta(seconds=timeout)
-            hosts = hosts.filter(ansible_facts_modified__gte=timeout)
+            hosts = self._get_inventory_hosts(ansible_facts_modified__gte=timeout)
+        else:
+            hosts = self._get_inventory_hosts()
         for host in hosts:
             filepath = os.sep.join(map(str, [destination, host.name]))
             if not os.path.realpath(filepath).startswith(destination):


Test patch: