Repository: tellor-io/telliot-feeds (language: python)

Issue:
Fix Profitability Calculation? 
Problem encountered when trying to listen for Autopay tips on mainnet. 

Expected behavior: telliot listens for tips and submits when it's profitable -gas costs.
observed behavior: telliot submits immediately for the small time based reward.

Start command: `telliot-feeds -a main report -p 100 -wp 600`

Versions: 
telliot-core commit hash #311909100be6110418bd08a5a1dacd340fbc145d
telliot-feeds commit hash #02647ca5b2c2db2e6e2142bc89ee9c4c49c62bc1

log:
```
INFO    | telliot_feeds.sources.price_aggregator | Number of Sources used for this report are: 2
INFO    | telliot_feeds.reporters.tellorflex |
                tips: 0.23333333333333334 TRB
                gas limit: 350000
                legacy gas price: 53

INFO    | telliot_feeds.reporters.tellorflex | Estimated profit: $2.71
INFO    | telliot_feeds.reporters.tellorflex | tip price: 2.73, gas costs: 0.01749265
INFO    | telliot_feeds.reporters.tellorflex | Estimated percent profit: 15513.22%
INFO    | telliot_feeds.sources.price_aggregator | Running median on [16797.66, 16788.64, 16773.02, 16776.14, 16777.3]
INFO    | telliot_feeds.sources.price_aggregator | Feed Price: 16777.3 reported at time 2022-11-15 19:52:21.212517+00:00
INFO    | telliot_feeds.sources.price_aggregator | Number of Sources used for this report are: 5
DEBUG   | telliot_feeds.reporters.interval | Sending submitValue transaction
INFO    | telliot_feeds.reporters.interval | View reported data:
https://etherscan.io/tx/NNNHHIJIJKJOJKHLKH
```


Discussion / hints:
reproduced:
```console
$ telliot-feeds -a personal-2 report -p 100 -wp 600
INFO    | telliot_core | telliot-core 0.1.7dev0
INFO    | telliot_core | Connected to eth-mainnet [default account: personal-2], time: 2022-11-23 06:56:18.783310
Your current settings...
Your chain id: 1

Your mainnet endpoint: 
 - provider: Infura
 - RPC url: wss://mainnet.infura.io/ws/v3/****
 - explorer url: https://etherscan.io
Your account: personal-2 at address ****
Proceed with current settings (y) or update (n)? [Y/n]: 
Enter password for personal-2 account: 

Reporting with synchronized queries
Current chain ID: 1
Expected percent profit: 100.0%
Transaction type: 0
Gas Limit: 350000
Legacy gas price (gwei): None
Max fee (gwei): None
Priority fee (gwei): None
Gas price speed: fast
Desired stake amount: 10.0
Minimum native token balance: 0.25 ETH


Press [ENTER] to confirm settings.
INFO    | telliot_feeds.reporters.tellorflex | Reporting with account: ****
INFO    | telliot_feeds.reporters.tellor_360 | Current Oracle stakeAmount: 130.15184381778744
INFO    | telliot_feeds.reporters.tellor_360 | 
            STAKER INFO
            start date: 0
            stake_balance: 0.0
            locked_balance: 0
            last report: 0
            reports count: 0
            
INFO    | telliot_feeds.reporters.tellor_360 | Approving and depositing stake...
INFO    | telliot_feeds.reporters.tellor_360 | Current wallet TRB balance: ****
Enter encryption password for personal-2: 
INFO    | telliot_core.contract.contract | approve transaction succeeded. (https://etherscan.io/tx/****)
INFO    | telliot_core.contract.contract | depositStake transaction succeeded. (https://etherscan.io/tx/****)
INFO    | telliot_feeds.reporters.tips.listener.funded_feeds | No funded feeds returned by autopay function call
INFO    | telliot_feeds.reporters.interval | Current query: {"type":"SpotPrice","asset":"sushi","currency":"usd"}
ERROR   | telliot_feeds.sources.price.spot.bittrex | Unable to decode Bittrex JSON
INFO    | telliot_feeds.sources.price_aggregator | Running median on [0.858546, 0.8536, 0.8524, 0.85154, 0.8518]
INFO    | telliot_feeds.sources.price_aggregator | Feed Price: 0.8524 reported at time 2022-11-23 12:57:13.676513+00:00
INFO    | telliot_feeds.sources.price_aggregator | Number of sources used in aggregate: 5
INFO    | telliot_feeds.sources.price_aggregator | Running median on [11.25, 11.2]
INFO    | telliot_feeds.sources.price_aggregator | Feed Price: 11.225 reported at time 2022-11-23 12:57:13.676733+00:00
INFO    | telliot_feeds.sources.price_aggregator | Number of sources used in aggregate: 2
INFO    | telliot_feeds.reporters.tellorflex | 
                tips: 1.0 TRB
                gas limit: 350000
                legacy gas price: 53
                
INFO    | telliot_feeds.reporters.tellorflex | Estimated profit: $11.21
INFO    | telliot_feeds.reporters.tellorflex | tip price: 11.22, gas costs: 0.015812020000000003
INFO    | telliot_feeds.reporters.tellorflex | Estimated percent profit: 70890.3%
INFO    | telliot_feeds.reporters.interval | TESTING...
INFO    | telliot_feeds.reporters.interval | Sleeping for 600 seconds
```
I think I found the problem:
- Tellor360 reporter inherits from TellorFlex reporter
- Tellor360 is using the profit calculation of TellorFlex reporter
- TellorFlex reporter's `ensure_profitable` method is using the price of MATIC, even though we're trying to report on ethereum mainnet:
```python
# Fetch token prices in USD
price_feeds = [matic_usd_median_feed, trb_usd_median_feed]
_ = await asyncio.gather(*[feed.source.fetch_new_datapoint() for feed in price_feeds])
price_matic_usd = matic_usd_median_feed.source.latest[0]
...
costs_usd = costs / 1e9 * price_matic_usd
...
```

Fix patch:
diff --git a/src/telliot_feeds/cli/commands/report.py b/src/telliot_feeds/cli/commands/report.py
index 45c087aa..0a982792 100644
--- a/src/telliot_feeds/cli/commands/report.py
+++ b/src/telliot_feeds/cli/commands/report.py
@@ -26,7 +26,7 @@
 from telliot_feeds.reporters.flashbot import FlashbotsReporter
 from telliot_feeds.reporters.rng_interval import RNGReporter
 from telliot_feeds.reporters.tellor_360 import Tellor360Reporter
-from telliot_feeds.reporters.tellorflex import TellorFlexReporter
+from telliot_feeds.reporters.tellor_flex import TellorFlexReporter
 from telliot_feeds.utils.cfg import check_endpoint
 from telliot_feeds.utils.cfg import setup_config
 from telliot_feeds.utils.log import get_logger
@@ -36,7 +36,6 @@
 
 logger = get_logger(__name__)
 
-TELLOR_X_CHAINS = (1, 4, 5)
 
 STAKE_MESSAGE = (
     "\U00002757Telliot will automatically stake more TRB "
diff --git a/src/telliot_feeds/cli/utils.py b/src/telliot_feeds/cli/utils.py
index 9ba6fa08..19655d8b 100644
--- a/src/telliot_feeds/cli/utils.py
+++ b/src/telliot_feeds/cli/utils.py
@@ -16,13 +16,12 @@
 from telliot_core.apps.core import TelliotCore
 from telliot_core.cli.utils import cli_core
 
+from telliot_feeds.constants import DIVA_PROTOCOL_CHAINS
 from telliot_feeds.datafeed import DataFeed
 from telliot_feeds.feeds import DATAFEED_BUILDER_MAPPING
 from telliot_feeds.queries.abi_query import AbiQuery
 
 
-DIVA_PROTOCOL_CHAINS = (137, 80001, 3, 5)
-
 load_dotenv()
 
 
diff --git a/src/telliot_feeds/constants.py b/src/telliot_feeds/constants.py
new file mode 100644
index 00000000..f30e7299
--- /dev/null
+++ b/src/telliot_feeds/constants.py
@@ -0,0 +1,5 @@
+DIVA_PROTOCOL_CHAINS = (137, 80001, 3, 5)
+
+POLYGON_CHAINS = {137, 80001}
+
+ETHEREUM_CHAINS = {1, 3, 5}
diff --git a/src/telliot_feeds/reporters/tellor_360.py b/src/telliot_feeds/reporters/tellor_360.py
index c2caa3d1..9909258c 100644
--- a/src/telliot_feeds/reporters/tellor_360.py
+++ b/src/telliot_feeds/reporters/tellor_360.py
@@ -13,7 +13,7 @@
 from telliot_feeds.feeds import CATALOG_FEEDS
 from telliot_feeds.feeds import DataFeed
 from telliot_feeds.reporters.rewards.time_based_rewards import get_time_based_rewards
-from telliot_feeds.reporters.tellorflex import TellorFlexReporter
+from telliot_feeds.reporters.tellor_flex import TellorFlexReporter
 from telliot_feeds.reporters.tips.suggest_datafeed import get_feed_and_tip
 from telliot_feeds.reporters.tips.tip_amount import fetch_feed_tip
 from telliot_feeds.utils.log import get_logger
diff --git a/src/telliot_feeds/reporters/tellorflex.py b/src/telliot_feeds/reporters/tellor_flex.py
similarity index 96%
rename from src/telliot_feeds/reporters/tellorflex.py
rename to src/telliot_feeds/reporters/tellor_flex.py
index d6755a98..e36cad7f 100644
--- a/src/telliot_feeds/reporters/tellorflex.py
+++ b/src/telliot_feeds/reporters/tellor_flex.py
@@ -17,13 +17,13 @@
 
 from telliot_feeds.datafeed import DataFeed
 from telliot_feeds.feeds import CATALOG_FEEDS
-from telliot_feeds.feeds.matic_usd_feed import matic_usd_median_feed
 from telliot_feeds.feeds.trb_usd_feed import trb_usd_median_feed
 from telliot_feeds.reporters.interval import IntervalReporter
 from telliot_feeds.reporters.reporter_autopay_utils import autopay_suggested_report
 from telliot_feeds.reporters.reporter_autopay_utils import CATALOG_QUERY_IDS
 from telliot_feeds.reporters.reporter_autopay_utils import get_feed_tip
 from telliot_feeds.utils.log import get_logger
+from telliot_feeds.utils.reporter_utils import get_native_token_feed
 from telliot_feeds.utils.reporter_utils import tellor_suggested_report
 
 
@@ -82,7 +82,9 @@ def __init__(
         assert self.acct_addr == to_checksum_address(self.account.address)
 
     async def fetch_gas_price(self, speed: Optional[str] = None) -> Optional[int]:
-        """Fetch estimated gas prices for Polygon mainnet."""
+        """Fetch estimated gas prices.
+
+        Expected to return gas price in gwei."""
         return await legacy_gas_station(chain_id=self.chain_id, speed=speed)  # type: ignore
 
     async def in_dispute(self, new_stake_amount: Any) -> bool:
@@ -268,11 +270,12 @@ async def ensure_profitable(
         status = ResponseStatus()
         tip = self.autopaytip
         # Fetch token prices in USD
-        price_feeds = [matic_usd_median_feed, trb_usd_median_feed]
+        native_token_feed = get_native_token_feed(self.chain_id)
+        price_feeds = [native_token_feed, trb_usd_median_feed]
         _ = await asyncio.gather(*[feed.source.fetch_new_datapoint() for feed in price_feeds])
-        price_matic_usd = matic_usd_median_feed.source.latest[0]
+        price_native_token = native_token_feed.source.latest[0]
         price_trb_usd = trb_usd_median_feed.source.latest[0]
-        if price_matic_usd is None or price_trb_usd is None:
+        if price_native_token is None or price_trb_usd is None:
             logger.warning("Unable to fetch token price")
             return None
 
@@ -305,7 +308,7 @@ async def ensure_profitable(
                 """
             )
 
-            costs = self.gas_limit * self.max_fee
+            costs = self.gas_limit * self.max_fee  # in gwei
 
         # Using transaction type 0 (legacy)
         else:
@@ -329,7 +332,7 @@ async def ensure_profitable(
 
         # Calculate profit
         rev_usd = tip / 1e18 * price_trb_usd
-        costs_usd = costs / 1e9 * price_matic_usd
+        costs_usd = costs / 1e9 * price_native_token  # convert gwei costs to eth, then to usd
         profit_usd = rev_usd - costs_usd
         logger.info(f"Estimated profit: ${round(profit_usd, 2)}")
         logger.info(f"tip price: {round(rev_usd, 2)}, gas costs: {costs_usd}")
diff --git a/src/telliot_feeds/sources/etherscan_gas.py b/src/telliot_feeds/sources/etherscan_gas.py
index fde63d61..76d962fb 100644
--- a/src/telliot_feeds/sources/etherscan_gas.py
+++ b/src/telliot_feeds/sources/etherscan_gas.py
@@ -40,7 +40,9 @@ class EtherscanGasPriceSource(DataSource[EtherscanGasPrice]):
     api_key: str = ""
 
     async def fetch_new_datapoint(self) -> OptionalDataPoint[EtherscanGasPrice]:
-        """Fetch new value and store it for later retrieval"""
+        """Fetch new value and store it for later retrieval
+
+        Gase prices are returned in gwei."""
 
         headers = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:77.0) Gecko/20190101 Firefox/77.0"}
         msg = "https://api.etherscan.io/api?module=gastracker&action=gasoracle"
diff --git a/src/telliot_feeds/utils/reporter_utils.py b/src/telliot_feeds/utils/reporter_utils.py
index 6be70315..872abfe1 100644
--- a/src/telliot_feeds/utils/reporter_utils.py
+++ b/src/telliot_feeds/utils/reporter_utils.py
@@ -16,6 +16,11 @@
 from telliot_core.tellor.tellorx.oracle import TellorxOracleContract
 from web3 import Web3
 
+from telliot_feeds.constants import ETHEREUM_CHAINS
+from telliot_feeds.constants import POLYGON_CHAINS
+from telliot_feeds.datafeed import DataFeed
+from telliot_feeds.feeds.eth_usd_feed import eth_usd_median_feed
+from telliot_feeds.feeds.matic_usd_feed import matic_usd_median_feed
 from telliot_feeds.queries.query_catalog import query_catalog
 from telliot_feeds.utils.log import get_logger
 
@@ -142,3 +147,13 @@ def prompt_for_abi() -> Any:
             abi = json.load(f)
         return abi
     return None
+
+
+def get_native_token_feed(chain_id: int) -> DataFeed[float]:
+    """Return native token feed for a given chain ID."""
+    if chain_id in ETHEREUM_CHAINS:
+        return eth_usd_median_feed
+    elif chain_id in POLYGON_CHAINS:
+        return matic_usd_median_feed
+    else:
+        raise ValueError(f"Cannot fetch native token feed. Invalid chain ID: {chain_id}")


Test patch:
diff --git a/tests/reporters/test_360reporter.py b/tests/reporters/test_360_reporter.py
similarity index 100%
rename from tests/reporters/test_360reporter.py
rename to tests/reporters/test_360_reporter.py
diff --git a/tests/reporters/test_flex_reporter.py b/tests/reporters/test_flex_reporter.py
index ddaeb0e0..4d1f4f12 100644
--- a/tests/reporters/test_flex_reporter.py
+++ b/tests/reporters/test_flex_reporter.py
@@ -11,7 +11,7 @@
 from telliot_feeds.feeds import CATALOG_FEEDS
 from telliot_feeds.feeds.eth_usd_feed import eth_usd_median_feed
 from telliot_feeds.feeds.matic_usd_feed import matic_usd_median_feed
-from telliot_feeds.reporters.tellorflex import TellorFlexReporter
+from telliot_feeds.reporters.tellor_flex import TellorFlexReporter
 
 
 @pytest_asyncio.fixture(scope="function")
@@ -62,11 +62,20 @@ async def test_YOLO_feed_suggestion(tellor_flex_reporter):
 
 @pytest.mark.asyncio
 async def test_ensure_profitable(tellor_flex_reporter):
-    status = await tellor_flex_reporter.ensure_profitable(matic_usd_median_feed)
+    r = tellor_flex_reporter
+    r.expected_profit = "YOLO"
+    unused_feed = matic_usd_median_feed
+    status = await r.ensure_profitable(unused_feed)
 
     assert isinstance(status, ResponseStatus)
     assert status.ok
 
+    r.chain_id = 1
+    r.expected_profit = 100.0
+    status = await r.ensure_profitable(unused_feed)
+
+    assert not status.ok
+
 
 @pytest.mark.asyncio
 async def test_fetch_gas_price(tellor_flex_reporter):
diff --git a/tests/reporters/test_reporter_utils.py b/tests/reporters/test_reporter_utils.py
deleted file mode 100644
index 005aafd8..00000000
--- a/tests/reporters/test_reporter_utils.py
+++ /dev/null
@@ -1,11 +0,0 @@
-import pytest
-
-from telliot_feeds.utils.reporter_utils import is_online
-
-
-@pytest.mark.asyncio
-async def test_checking_if_online():
-    """test telliot check for internet connection"""
-
-    online = await is_online()
-    assert isinstance(online, bool)
diff --git a/tests/test_bct_usd.py b/tests/test_bct_usd.py
index 76f7cd62..482686b0 100644
--- a/tests/test_bct_usd.py
+++ b/tests/test_bct_usd.py
@@ -4,7 +4,7 @@
 from web3.datastructures import AttributeDict
 
 from telliot_feeds.feeds.bct_usd_feed import bct_usd_median_feed
-from telliot_feeds.reporters.tellorflex import TellorFlexReporter
+from telliot_feeds.reporters.tellor_flex import TellorFlexReporter
 
 
 @pytest.mark.asyncio
diff --git a/tests/test_dai_usd.py b/tests/test_dai_usd.py
index 13fec1ec..97d58b42 100644
--- a/tests/test_dai_usd.py
+++ b/tests/test_dai_usd.py
@@ -4,7 +4,7 @@
 from web3.datastructures import AttributeDict
 
 from telliot_feeds.feeds.dai_usd_feed import dai_usd_median_feed
-from telliot_feeds.reporters.tellorflex import TellorFlexReporter
+from telliot_feeds.reporters.tellor_flex import TellorFlexReporter
 
 
 @pytest.mark.asyncio
diff --git a/tests/test_numeric_api_response_feed.py b/tests/test_numeric_api_response_feed.py
index 2a6aafd2..ab649ef2 100644
--- a/tests/test_numeric_api_response_feed.py
+++ b/tests/test_numeric_api_response_feed.py
@@ -5,7 +5,7 @@
 
 from telliot_feeds.datafeed import DataFeed
 from telliot_feeds.queries.numeric_api_response_query import NumericApiResponse
-from telliot_feeds.reporters.tellorflex import TellorFlexReporter
+from telliot_feeds.reporters.tellor_flex import TellorFlexReporter
 from telliot_feeds.sources.numeric_api_response import NumericApiResponseSource
 
 url = "https://taylorswiftapi.herokuapp.com/get"
diff --git a/tests/test_olympus.py b/tests/test_olympus.py
index d3815151..008763a8 100644
--- a/tests/test_olympus.py
+++ b/tests/test_olympus.py
@@ -4,7 +4,7 @@
 from web3.datastructures import AttributeDict
 
 from telliot_feeds.feeds.olympus import ohm_eth_median_feed
-from telliot_feeds.reporters.tellorflex import TellorFlexReporter
+from telliot_feeds.reporters.tellor_flex import TellorFlexReporter
 
 
 @pytest.mark.asyncio
diff --git a/tests/test_vesq.py b/tests/test_vesq.py
index 2f84205c..b7803d23 100644
--- a/tests/test_vesq.py
+++ b/tests/test_vesq.py
@@ -4,7 +4,7 @@
 from web3.datastructures import AttributeDict
 
 from telliot_feeds.feeds.vesq import vsq_usd_median_feed
-from telliot_feeds.reporters.tellorflex import TellorFlexReporter
+from telliot_feeds.reporters.tellor_flex import TellorFlexReporter
 
 
 @pytest.mark.asyncio
diff --git a/tests/utils/test_reporter_utils.py b/tests/utils/test_reporter_utils.py
index dc203804..b23818a7 100644
--- a/tests/utils/test_reporter_utils.py
+++ b/tests/utils/test_reporter_utils.py
@@ -9,10 +9,14 @@
 from web3 import Web3
 
 from telliot_feeds.cli.main import main as cli_main
+from telliot_feeds.feeds.eth_usd_feed import eth_usd_median_feed
+from telliot_feeds.feeds.matic_usd_feed import matic_usd_median_feed
 from telliot_feeds.queries.query import OracleQuery
 from telliot_feeds.queries.query_catalog import query_catalog
 from telliot_feeds.utils.log import get_logger
+from telliot_feeds.utils.reporter_utils import get_native_token_feed
 from telliot_feeds.utils.reporter_utils import has_native_token_funds
+from telliot_feeds.utils.reporter_utils import is_online
 from telliot_feeds.utils.reporter_utils import reporter_sync_schedule
 from telliot_feeds.utils.reporter_utils import tellor_suggested_report
 
@@ -89,3 +93,23 @@ def fail_balance_read(*args, **kwargs):
             assert has_funds is False
             assert "bango" in caplog.text
             assert f"Error fetching native token balance for {addr}" in caplog.text
+
+
+def test_get_native_token_feed():
+    """Test get_native_token_feed"""
+    f = get_native_token_feed(1)
+    assert f == eth_usd_median_feed
+
+    f = get_native_token_feed(137)
+    assert f == matic_usd_median_feed
+
+    with pytest.raises(ValueError):
+        _ = get_native_token_feed(0)
+
+
+@pytest.mark.asyncio
+async def test_checking_if_online():
+    """test telliot check for internet connection"""
+
+    online = await is_online()
+    assert isinstance(online, bool)