Repository: home-assistant/core
Pull request: Fix `pulse counter frequency` sensors for Shelly Plus Uni
<!--
  You are amazing! Thanks for contributing to our project!
  Please, DO NOT DELETE ANY TEXT from this template! (unless instructed).
-->
## Breaking change
<!--
  If your PR contains a breaking change for existing users, it is important
  to tell them what breaks, how to make it work again and why we did this.
  This piece of text is published with the release notes, so it helps if you
  write it towards our users, not us.
  Note: Remove this section if this PR is NOT a breaking change.
-->


## Proposed change
<!--
  Describe the big picture of your changes here to communicate to the
  maintainers why we should accept this pull request. If it fixes a bug
  or resolves a feature request, be sure to link to that issue in the
  additional information section.
-->
This fixes a mistake I made here https://github.com/home-assistant/core/pull/119898

I also add a test for disabled `xfreq` sensor.

Also improved 'remove_condition' for other input sensors.


## Type of change
<!--
  What type of change does your PR introduce to Home Assistant?
  NOTE: Please, check only 1! box!
  If your PR requires multiple boxes to be checked, you'll most likely need to
  split it into multiple PRs. This makes things easier and faster to code review.
-->

- [ ] Dependency upgrade
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New integration (thank you!)
- [ ] New feature (which adds functionality to an existing integration)
- [ ] Deprecation (breaking change to happen in the future)
- [ ] Breaking change (fix/feature causing existing functionality to break)
- [ ] Code quality improvements to existing code or addition of tests

## Additional information
<!--
  Details are important, and help maintainers processing your PR.
  Please be sure to fill out additional details, if applicable.
-->

- This PR fixes or closes issue: fixes #121124
- This PR is related to issue: 
- Link to documentation pull request: 

## Checklist
<!--
  Put an `x` in the boxes that apply. You can also fill these out after
  creating the PR. If you're unsure about any of them, don't hesitate to ask.
  We're here to help! This is simply a reminder of what we are going to look
  for before merging your code.
-->

- [ ] The code change is tested and works locally.
- [x] Local tests pass. **Your PR cannot be merged unless tests pass**
- [x] There is no commented out code in this PR.
- [x] I have followed the [development checklist][dev-checklist]
- [x] I have followed the [perfect PR recommendations][perfect-pr]
- [x] The code has been formatted using Ruff (`ruff format homeassistant tests`)
- [x] Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

- [ ] Documentation added/updated for [www.home-assistant.io][docs-repository]

If the code communicates with devices, web services, or third-party tools:

- [ ] The [manifest file][manifest-docs] has all fields filled out correctly.  
      Updated and included derived files by running: `python3 -m script.hassfest`.
- [ ] New or updated dependencies have been added to `requirements_all.txt`.  
      Updated by running `python3 -m script.gen_requirements_all`.
- [ ] For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
- [ ] Untested files have been added to `.coveragerc`.

<!--
  This project is very active and we have a high turnover of pull requests.

  Unfortunately, the number of incoming pull requests is higher than what our
  reviewers can review and merge so there is a long backlog of pull requests
  waiting for review. You can help here!
  
  By reviewing another pull request, you will help raise the code quality of
  that pull request and the final review will be faster. This way the general
  pace of pull request reviews will go up and your wait time will go down.
  
  When picking a pull request to review, try to choose one that hasn't yet
  been reviewed.

  Thanks for helping out!
-->

To help with the load of incoming pull requests:

- [ ] I have reviewed two other [open pull requests][prs] in this repository.

[prs]: https://github.com/home-assistant/core/pulls?q=is%3Aopen+is%3Apr+-author%3A%40me+-draft%3Atrue+-label%3Awaiting-for-upstream+sort%3Acreated-desc+review%3Anone+-status%3Afailure

<!--
  Thank you for contributing <3

  Below, some useful links you could explore:
-->
[dev-checklist]: https://developers.home-assistant.io/docs/development_checklist/
[manifest-docs]: https://developers.home-assistant.io/docs/creating_integration_manifest/
[quality-scale]: https://developers.home-assistant.io/docs/integration_quality_scale_index/
[docs-repository]: https://github.com/home-assistant/home-assistant.io
[perfect-pr]: https://developers.home-assistant.io/docs/review-process/#creating-the-perfect-pr


Problem statement:
Shelly PlusUni Output switches lose connection after Core Update 2024.7.0
### The problem

I have a Shelly Plus Uni that I use its Outputs for to open the door of my apartment building every day.

Right after todays core update to 2024.7.0 the Outputs can be toggled on, but not toggled off, as they become unavailable right after turning them on.

Reverting back to 2024.6.4 solves the problem.

### What version of Home Assistant Core has the issue?

2024.6.4

### What was the last working version of Home Assistant Core?

2024.6.4

### What type of installation are you running?

Home Assistant OS

### Integration causing the issue

Shelly

### Link to integration documentation on our website

_No response_

### Diagnostics information

_No response_

### Example YAML snippet

_No response_

### Anything in the logs that might be useful for us?

_No response_

### Additional information

_No response_

Hints from issue discussion:
Hey there @balloob, @bieniu, @thecode, @chemelli74, @bdraco, mind taking a look at this issue as it has been labeled with an integration (`shelly`) you are listed as a [code owner](https://github.com/home-assistant/core/blob/dev/CODEOWNERS#L1263) for? Thanks!

<details>
<summary>Code owner commands</summary>


Code owners of `shelly` can trigger bot actions by commenting:

- `@home-assistant close` Closes the issue.
- `@home-assistant rename Awesome new title` Renames the issue.
- `@home-assistant reopen` Reopen the issue.
- `@home-assistant unassign shelly` Removes the current integration label and assignees on the issue, add the integration domain after the command.
- `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
- `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

</details>


<sub><sup>(message by CodeOwnersMention)</sup></sub>

---

[shelly documentation](https://www.home-assistant.io/integrations/shelly)
[shelly source](https://github.com/home-assistant/core/tree/dev/homeassistant/components/shelly)
<sub><sup>(message by IssueLinks)</sup></sub>
Please attach a debug log file and diagnostics file.
Can you explain how I do that? I'm new to all of this and would like to help, as good as I can.
Add this to your `configuration.yaml` file:
```yaml
logger:
  default: warning
  logs:
    aioshelly: debug
    homeassistant.components.shelly: debug
```
restart HA, wait half an hour and attach here a file `/config/home-assistant.log`.

Diagnostics file, look here https://www.home-assistant.io/integrations/diagnostics/

same problem
Logger: homeassistant.components.shelly
Source: helpers/update_coordinator.py:312
integration: Shelly ([documentation](https://www.home-assistant.io/integrations/shelly), [issues](https://github.com/home-assistant/core/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+shelly%22))
First occurred: 11:11:00 PM (9 occurrences)
Last logged: 11:19:02 PM

Unexpected error fetching shellyplusuni-e465b84718e4 data
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 312, in _async_refresh
    self.data = await self._async_update_data()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 594, in _async_update_data
    if not await self._async_device_connect_task():
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 167, in _async_device_connect_task
    await self.device.initialize()
  File "/usr/local/lib/python3.12/site-packages/aioshelly/rpc_device/device.py", line 208, in initialize
    self._update_listener(self, RpcUpdateType.INITIALIZED)
  File "/usr/src/homeassistant/homeassistant/components/shelly/coordinator.py", line 680, in _async_handle_update
    self.async_set_updated_data(None)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 447, in async_set_updated_data
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 165, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/components/shelly/entity.py", line 372, in _update_callback
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1007, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1133, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1068, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1013, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 533, in state
    value = self.native_value
            ^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/sensor.py", line 1097, in native_value
    return self.attribute_value
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/entity.py", line 519, in attribute_value
    self._last_value = self.entity_description.value(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/sensor.py", line 998, in <lambda>
    value=lambda status, _: status["freq"],
                            ~~~~~~^^^^^^^^
KeyError: 'freq'
I have the same errors with the Shelly Plus Uni, my workaround:

Disable all entity end with “pulse_counter_frequency” and relaod device or integration.
![image](https://github.com/home-assistant/core/assets/125592865/222c27ba-a630-42fc-afb3-2bf93f325cb9)

```
Logger: homeassistant.components.sensor
Quelle: helpers/entity_platform.py:598
Integration: Sensor (Dokumentation, Probleme)
Erstmals aufgetreten: 07:30:39 (5 Vorkommnisse)
Zuletzt protokolliert: 07:30:40

Error adding entity sensor.ku_lights_fairylights_input_2_pulse_counter_frequency for domain sensor with platform shelly
Error adding entity sensor.bo_itserver_options_input_2_pulse_counter_frequency for domain sensor with platform shelly
Error adding entity sensor.thug_housebell_input_2_pulse_counter_frequency for domain sensor with platform shelly
Error adding entity sensor.bo_itserver_powermgmt_input_2_pulse_counter_frequency for domain sensor with platform shelly
Error adding entity sensor.sboitproxmoxpower_input_2_pulse_counter_frequency for domain sensor with platform shelly
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 598, in _async_add_entities
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 912, in _async_add_entity
    await entity.add_to_platform_finish()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1362, in add_to_platform_finish
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1007, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1133, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1068, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1013, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 533, in state
    value = self.native_value
            ^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/sensor.py", line 1097, in native_value
    return self.attribute_value
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/entity.py", line 519, in attribute_value
    self._last_value = self.entity_description.value(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/shelly/sensor.py", line 998, in <lambda>
    value=lambda status, _: status["freq"],
                            ~~~~~~^^^^^^^^
KeyError: 'freq'
```

Disabling the pulse_counter_frequency entities does not help for me, I deactivated them month ago as I only use the output (0) entity and nothing else. 
Hello,
I have the same problem. Everything becomes unavailable after a few second
> I have the same errors with the Shelly Plus Uni, my workaround:
> 
> Disable all entity end with “pulse_counter_frequency” and relaod device or integration

Thanks, your workaround worked for me too.

Commit message: Fix Pulse counter frequency sensors for Shelly Uni

Patch to review:
diff --git a/homeassistant/components/shelly/sensor.py b/homeassistant/components/shelly/sensor.py
index 743c7c7ff01e3..b41cb090b9827 100644
--- a/homeassistant/components/shelly/sensor.py
+++ b/homeassistant/components/shelly/sensor.py
@@ -991,20 +991,18 @@ class RestSensorDescription(RestEntityDescription, SensorEntityDescription):
     ),
     "counter_frequency": RpcSensorDescription(
         key="input",
-        sub_key="counts",
+        sub_key="freq",
         name="Pulse counter frequency",
         native_unit_of_measurement=UnitOfFrequency.HERTZ,
         state_class=SensorStateClass.MEASUREMENT,
-        value=lambda status, _: status["freq"],
-        removal_condition=lambda config, status, key: (config[key]["enable"] is False),
+        removal_condition=lambda config, _, key: (config[key]["enable"] is False),
     ),
     "counter_frequency_value": RpcSensorDescription(
         key="input",
-        sub_key="counts",
+        sub_key="xfreq",
         name="Pulse counter frequency value",
-        value=lambda status, _: status["xfreq"],
         removal_condition=lambda config, status, key: (
-            config[key]["enable"] is False or status[key]["counts"].get("xfreq") is None
+            config[key]["enable"] is False or status[key].get("xfreq") is None
         ),
     ),
 }
diff --git a/tests/components/shelly/conftest.py b/tests/components/shelly/conftest.py
index 65ebdeb6996d1..e9662b55cf5cb 100644
--- a/tests/components/shelly/conftest.py
+++ b/tests/components/shelly/conftest.py
@@ -228,7 +228,9 @@ def mock_white_light_set_state(
     "input:1": {"id": 1, "percent": 89, "xpercent": 8.9},
     "input:2": {
         "id": 2,
-        "counts": {"total": 56174, "xtotal": 561.74, "freq": 208.00, "xfreq": 6.11},
+        "counts": {"total": 56174, "xtotal": 561.74},
+        "freq": 208.00,
+        "xfreq": 6.11,
     },
     "light:0": {"output": True, "brightness": 53.0},
     "light:1": {"output": True, "brightness": 53.0},
diff --git a/tests/components/shelly/test_sensor.py b/tests/components/shelly/test_sensor.py
index 513bcd875e2a1..c62a1f6f6ca92 100644
--- a/tests/components/shelly/test_sensor.py
+++ b/tests/components/shelly/test_sensor.py
@@ -828,3 +828,29 @@ async def test_rpc_pulse_counter_frequency_sensors(
     entry = entity_registry.async_get(entity_id)
     assert entry
     assert entry.unique_id == "123456789ABC-input:2-counter_frequency_value"
+
+
+async def test_rpc_disabled_xfreq(
+    hass: HomeAssistant,
+    mock_rpc_device: Mock,
+    entity_registry: EntityRegistry,
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    """Test RPC input with the xfreq sensor disabled."""
+    status = deepcopy(mock_rpc_device.status)
+    status["input:2"] = {
+        "id": 2,
+        "counts": {"total": 56174, "xtotal": 561.74},
+        "freq": 208.00,
+    }
+    monkeypatch.setattr(mock_rpc_device, "status", status)
+
+    await init_integration(hass, 2)
+
+    entity_id = f"{SENSOR_DOMAIN}.gas_pulse_counter_frequency_value"
+
+    state = hass.states.get(entity_id)
+    assert not state
+
+    entry = entity_registry.async_get(entity_id)
+    assert not entry


Code review comments:
diff_hunk: @@ -991,20 +991,18 @@ class RestSensorDescription(RestEntityDescription, SensorEntityDescription):
     ),
     "counter_frequency": RpcSensorDescription(
         key="input",
-        sub_key="counts",
+        sub_key="freq",
         name="Pulse counter frequency",
         native_unit_of_measurement=UnitOfFrequency.HERTZ,
         state_class=SensorStateClass.MEASUREMENT,
-        value=lambda status, _: status["freq"],
-        removal_condition=lambda config, status, key: (config[key]["enable"] is False),
+        removal_condition=lambda config, _, key: (config[key]["enable"] is False),
path: homeassistant/components/shelly/sensor.py
text: @user1:
Shouldn't we test for get('freq') is None like we do for xfreq ?

@author:
According to the documentation `freq` is always a number
![obraz](https://github.com/home-assistant/core/assets/478555/99adf272-2067-4f60-aa1a-4afc3e7967bb)
But I think we need here to check the type of the input. If I understand correctly `freq` is available only for the type `count`
https://shelly-api-docs.shelly.cloud/gen2/ComponentsAndServices/Input#status

@user1:
yup, `percent `and `xpercent` for **analog**, `freq` and `xfreq` for **count**