Repository: ME-ICA/tedana

Issue:
How to distribute more decision trees with tedana
### Summary

@BahmanTahayori is about to submit a manuscript for publication that uses a revision of the current decision tree. He asked me if we could add it to tedana `main` so that he can reference it in the tedana code. I think it's great to distribute more decision trees, particularly those that have been validated in manuscripts, but I want to do this in a way that won't cause future problems.

### Additional Detail

#### Concerns
1. We don't want to get in the business of judging who does or does not have a decision tree worthy of inclusion in tedana.
2. As we get more decision trees, the already cryptic names will get more cryptic. Bahman's is currently named `robust_tedana` (also the name of the overall method in the manuscript). Every new tree will be `[positive adjective]_tedana` and it will be hard to understand how to parse a long list of trees distributed with the code.

#### Proposed solutions

- For concern 1, my suggestion is to say we are willing to share any decision tree that is included in a manuscript that includes some validations against other decision trees or other methods. 
- For concern 2
  - I briefly spoke to @eurunuela and he suggested, having a place outside of the repository to share decision trees. If a decision tree isn't in the tedana repo, the code could query that repo and download the appropriate decision tree. We are already storing decision trees on figshare and have links to the json files. For example tedana_orig ( https://figshare.com/articles/code/Component_selection_decision_trees_in_tedana/25251433/2?file=45211096 ). The nice part of this approach is that figshare also have versioning so we can have multiple versions of the same trees that could be consistently rerun in the future. The minus is that this is. one more complexity to add to the code.
  - An intermediate option would be to make a subfolder of https://github.com/ME-ICA/tedana/tree/main/tedana/resources/decision_trees for user-provided trees. They'd just need distinct names and we already have a system for each one to include the reference for where it was used. They wouldn't appear in the help menu, but anyone could call them by their name. The `demo_` trees in the above folder are already like this in that we're distributing them, but they are easy to run, but aren't highlighted in the help menus.
  - For naming, if there's some division between the main ones and others, then we can be very flexible. If they are all together, we might want to lean into the original `kundu` tree and use first-author names, i.e. `tahayori` for their new tree.

### Next Steps

- Acutely @BahmanTahayori wants his json file stored somewhere consistent that he can reference in his publication he wants to submit in a couple of days. I can very quickly [ut it into a new version3 of our figshare page and then he has a clear reference. I think he'd prefer I just add it to the tedana repo. I'm fine either or both, but this is a group project and I don't want to make this decision solo. $\Large\color{red}{\textbf{I'd like others to give input on this ASAP.}}$
- Decide on a medium term approach. Proposals above are:
  - More more stuff in https://github.com/ME-ICA/tedana/tree/main/tedana/resources/decision_trees with some rule on what we'd add and some naming convention
  -  Put more trees in something like https://github.com/ME-ICA/tedana/tree/main/tedana/resources/decision_trees/published_trees with less concern for a naming convention. We could then decide to promote trees to the main directory based on usage and validated improvements over multiple publications
  - Set up a system to automatically download trees from figshare or OSF.

I think @eurunuela's suggestion to have a download system is the most elegant, but would require more time coding. I am currently leaning towards the subdirectory option with all trees also being added to our figshare site. That said, I should not make this decision solo. I'm tagging others who are or might be playing with decision trees because this might affect you & you might have opinion: @goodalse2019 @n-reddy @tsalo @CesarCaballeroGaudes @dowdlelt @smoia @mmedina002 @bf777

Hints from issue comments:
@tsalo: For concern 1, I think we should require a preprint for any decision tree we don't label as an example/demo. Plus, the decision tree can link back to the preprint DOI. For concern 2, I like having figshare + a fetcher function, but putting it in the repo is fine by me too. I do think we should name them using the first author and preprint/publication date so that it's clear where each one comes from and so we don't invite one-upsmanship in the naming.@bf777: Having more options for decision trees is great, so long as they are clearly labelled. Having the additional trees be hosted externally is an option, and a solution that I have used in the past with optional model downloads for my own software packages. If a script is added that 1) calls an API or fetcher from e.g. figshare that seems stable and can automatically and painlessly download either all trees or a chosen tree; 2) ensures that trees that are downloaded are clearly labelled and traceable back to a DOI (as per @tsalo 's recommendation; and 3) run unobtrusively following tedana's installation, this is a solution that could offer more flexibility. Having the trees included directly in the repository could potentially create more headaches depending on how pushes are handled to the tedana repository (i.e. having to check code coverage every time a tree is added, overall more complex branching, etc.).@eurunuela: As Dan mentioned, I would store them in Figshare and retrieve them when Tedana runs. Users would provide the tree name via the CLI, and we would include a table in our documentation listing the different tree names and their corresponding papers.

I dislike the idea of collecting them in this repository. If people start sharing their decision trees, the repository could turn into a "decision tree store," which I believe is separate from this project. Additionally, including them in the repository makes me feel that we must validate the decision trees; storing them elsewhere would "detach" us from that responsibility.

This said, we probably need a new GitHub Issue template for decision tree submissions.@handwerkerd: Given the feedback so far, my current plan is to make a version 3 on figshare with @BahmanTahayori's tree, which I'll call `tahayori_robust_tedana` (or just `tahayori`). Then he'll have a permanent link to cite in his manuscript and we can either decide later to add to the main repo or create a function to fetch from figshare. If we revisit this later, we can just make a version 4 on figshare.@eurunuela: I would call it `tahayori_2025` as Taylor suggested.@handwerkerd: As I'm working on Bahman's tree, one quirk is that we might want to use the `\\citep` tag to cite new references. Currently, that means, even if we're not adding the new trees to the repo, we might need to expand the list of references. This is very do-able, but mildly annoying.@BahmanTahayori: Thank you, @handwerkerd, for getting this started. Hosting it on Figshare is a great idea. Personally, I’d prefer not to have my name included, but if the plan is to include contributors in the tree’s file name, I’m okay with that.@handwerkerd: The new decision tree is now publicly available here: https://figshare.com/articles/code/component_selection_decision_trees_in_tedana/25251433/3
Since @BahmanTahayori didn't want his name in the tree, we decided to name it `robust_tedana_2025` which is the beginning of the title of their publication & it's more reference-like. We can re-evaluate in the future, but a decision was needed to make sure this could be shared in parallel to submission.

I'm going to keep this issue open until we add functionality to pull decision trees from figshare (or decide on some other solution).

FWIW, the only author on figshare was "tedana community" I also added me & @tsalo because I'm hoping that might make it possible for someone besides me to have edit access. 

Solution patch:
diff --git a/docs/faq.rst b/docs/faq.rst
index c3fecde..d75e20a 100644
--- a/docs/faq.rst
+++ b/docs/faq.rst
@@ -161,6 +161,14 @@ If users are having trouble with PCA component estimation failing on a dataset,
 we recommend using RobustICA;
 and we invite users to send us feedback on its behavior and efficacy.
 
+Also note that ``--ica_method robustica`` can be paired with ``--n_robust_runs``.
+That second option defines how many interations of ICA to run.
+The default is ``--n_robust_runs 30`` which was selected because the resulting components
+stabilized and the activation volumes plateaued during testing on a large dataset.
+If one wants to examine on other data sets, a basic guide is,
+since larger ``--n_robust_runs`` means more compute time,
+it should be set to the lowest value where the stability of the resulting components plateaus.
+
 
 .. _MAPCA: https://github.com/ME-ICA/mapca
 .. _robustica: https://github.com/CRG-CNAG/robustica
diff --git a/tedana/decomposition/pca.py b/tedana/decomposition/pca.py
index 45801b1..79fbd0e 100644
--- a/tedana/decomposition/pca.py
+++ b/tedana/decomposition/pca.py
@@ -354,7 +354,7 @@ def tedpca(
         "d_table_score",
     ]
     # Even if user inputted, don't fit external_regressors to PCA components
-    component_table, _ = metrics.collect.generate_metrics(
+    component_table, comp_ts = metrics.collect.generate_metrics(
         data_cat=data_cat,
         data_optcom=data_optcom,
         mixing=comp_ts,
diff --git a/tedana/io.py b/tedana/io.py
index 30570e3..c92dd81 100644
--- a/tedana/io.py
+++ b/tedana/io.py
@@ -16,6 +16,7 @@ from typing import List
 import nibabel as nib
 import numpy as np
 import pandas as pd
+import requests
 from nilearn._utils import check_niimg
 from nilearn.image import new_img_like
 
@@ -502,6 +503,52 @@ def load_json(path: str) -> dict:
     return data
 
 
+def download_json(tree: str, out_dir: str) -> str:
+    """Download a json file from figshare unless the file already exists.
+
+    Parameters
+    ----------
+    tree : str
+        The name of the tree to download
+    out_dir : str
+        The directory where the json file will be saved
+
+    Returns
+    -------
+    save_path : str
+        The filepath of the downloaded decision tree.
+    """
+    base_url = "https://api.figshare.com/v2"
+    item_id = 25251433
+
+    fname = tree + ".json" if not tree.endswith(".json") else tree
+    save_path = op.join(out_dir, fname)
+
+    if op.isfile(save_path):
+        return save_path
+
+    try:
+        r = requests.get(f"{base_url}/articles/{item_id}")
+        r.raise_for_status()
+        metadata = r.json()
+
+        file_info = next((f for f in metadata["files"] if f["name"] == fname.lower()), None)
+
+        if not file_info:
+            return
+
+        download_r = requests.get(file_info["download_url"])
+        download_r.raise_for_status()
+
+        with open(save_path, "wb") as f:
+            f.write(download_r.content)
+        LGR.info(f"Tree {tree} downloaded from figshare to {save_path}")
+        return save_path
+
+    except requests.RequestException as e:
+        LGR.error(f"Cannot connect to figshare: {e}")
+
+
 def add_decomp_prefix(comp_num, prefix, max_value):
     """Create component name with leading zeros matching number of components.
 
diff --git a/tedana/metrics/collect.py b/tedana/metrics/collect.py
index 02b4a7d..d93eb72 100644
--- a/tedana/metrics/collect.py
+++ b/tedana/metrics/collect.py
@@ -35,7 +35,7 @@ def generate_metrics(
     external_regressors: Union[pd.DataFrame, None] = None,
     external_regressor_config: Union[List[Dict], None] = None,
     metrics: Union[List[str], None] = None,
-) -> Tuple[pd.DataFrame, Dict]:
+) -> Tuple[pd.DataFrame, npt.NDArray]:
     """Fit TE-dependence and -independence models to components.
 
     Parameters
@@ -76,9 +76,11 @@ def generate_metrics(
     component_table : (C x X) :obj:`pandas.DataFrame`
         Component metric table. One row for each component, with a column for each metric.
         The index is the component number.
-    external_regressor_config : :obj:`dict`
-        Info describing the external regressors and method used for fitting and statistical tests
-        (or None if none were inputed)
+    mixing : (T x C) array_like
+        Mixing matrix for converting input data to component space,
+        where `C` is components and `T` is the same as in `data_cat`
+        The signs of a component time series are flipped so that the components spatial map
+        has more positive voxel weights
     """
     # Load metric dependency tree from json file
     dependency_config = op.join(utils.get_resource_path(), "config", "metrics.json")
@@ -460,7 +462,7 @@ def generate_metrics(
     other_columns = [col for col in component_table.columns if col not in preferred_order]
     component_table = component_table[first_columns + other_columns]
 
-    return component_table, external_regressor_config
+    return component_table, mixing
 
 
 def get_metadata(component_table: pd.DataFrame) -> Dict:
@@ -665,7 +667,9 @@ def get_metadata(component_table: pd.DataFrame) -> Dict:
             "Description": (
                 "Optimal sign determined based on skew direction of component parameter estimates "
                 "across the brain. In cases where components were left-skewed (-1), the component "
-                "time series is flipped prior to metric calculation."
+                "time series and map weights are flipped prior to metric calculation. "
+                "This sign applies to the original mixing matrix and map weights. "
+                "The outputs produced by tedana are already flipped."
             ),
             "Levels": {
                 1: "Component is not flipped prior to metric calculation.",
diff --git a/tedana/reporting/static_figures.py b/tedana/reporting/static_figures.py
index eb9fdab..6f8682b 100644
--- a/tedana/reporting/static_figures.py
+++ b/tedana/reporting/static_figures.py
@@ -340,9 +340,6 @@ def comp_figures(ts, mask, component_table, mixing, io_generator, png_cmap):
     io_generator : :obj:`tedana.io.OutputGenerator`
         Output Generator object to use for this workflow
     """
-    # Flip signs of mixing matrix as needed
-    mixing = mixing * component_table["optimal sign"].values
-
     # regenerate the beta images
     component_maps_arr = stats.get_coeffs(ts, mixing, mask)
     component_maps_arr = component_maps_arr.reshape(
diff --git a/tedana/selection/component_selector.py b/tedana/selection/component_selector.py
index fd74cd4..789be2e 100644
--- a/tedana/selection/component_selector.py
+++ b/tedana/selection/component_selector.py
@@ -8,7 +8,7 @@ from typing import Dict, List, Union
 import pandas as pd
 from numpy import asarray
 
-from tedana.io import load_json
+from tedana.io import download_json, load_json
 from tedana.selection import selection_nodes
 from tedana.selection.selection_utils import (
     clean_dataframe,
@@ -43,19 +43,27 @@ class TreeError(Exception):
     pass
 
 
-def load_config(tree: str) -> Dict:
+def load_config(tree: str, out_dir: str = ".") -> Dict:
     """Load the json file with the decision tree and validate the fields in the decision tree.
 
     Parameters
     ----------
     tree : :obj:`str`
         The named tree or path to a JSON file that defines one
+    out_dir :obj:`str`, optional
+        The output directory. Default is current working directory.
 
     Returns
     -------
     tree : :obj:`dict`
         A validated decision tree for the component selection process.
+
+    Raises
+    ------
+    ValueError if the tree can't be loaded
     """
+    fname, dectree = None, None
+
     if tree in DEFAULT_TREES:
         fname = op.join(get_resource_path(), "decision_trees", tree + ".json")
     elif tree == "kundu":
@@ -65,20 +73,17 @@ def load_config(tree: str) -> Dict:
         )
         tree = "tedana_orig"
         fname = op.join(get_resource_path(), "decision_trees", tree + ".json")
-    else:
+    elif op.isfile(tree):
         fname = tree
+    else:
+        fname = download_json(tree, out_dir)
 
-    try:
+    if fname and op.isfile(fname):
         dectree = load_json(fname)
-    except FileNotFoundError:
+    else:
         raise ValueError(
             f"Cannot find tree {tree}. Please check your path or use a "
-            f"default tree ({DEFAULT_TREES})."
-        )
-    except IsADirectoryError:
-        raise ValueError(
-            f"Tree {tree} is a directory. Please supply a JSON file or "
-            f"default tree ({DEFAULT_TREES})."
+            f"default tree ({DEFAULT_TREES}) or one from figshare."
         )
 
     return validate_tree(dectree)
@@ -307,13 +312,15 @@ def validate_tree(tree: Dict) -> Dict:
 class ComponentSelector:
     """Load and classify components based on a specified ``tree``."""
 
-    def __init__(self, tree: str):
+    def __init__(self, tree: str, out_dir: str = "."):
         """Initialize the class using the info specified in the json file ``tree``.
 
         Parameters
         ----------
         tree : :obj:`str`
             The named tree or path to a JSON file that defines one.
+        out_dir :obj:`str`, optional
+            The output directory. Default is current working directory.
 
         Notes
         -----
@@ -321,7 +328,7 @@ class ComponentSelector:
         loads all information in the tree json file into ``ComponentSelector``.
         """
         self.tree_name = tree
-        self.tree = load_config(self.tree_name)
+        self.tree = load_config(self.tree_name, out_dir)
 
         LGR.info("Performing component selection with " + self.tree["tree_id"])
         LGR.info(self.tree.get("info", ""))
diff --git a/tedana/workflows/tedana.py b/tedana/workflows/tedana.py
index aa8fd8d..05c35a1 100644
--- a/tedana/workflows/tedana.py
+++ b/tedana/workflows/tedana.py
@@ -613,7 +613,7 @@ def tedana_workflow(
         data = [data]
 
     LGR.info("Initializing and validating component selection tree")
-    selector = ComponentSelector(tree)
+    selector = ComponentSelector(tree, out_dir)
 
     LGR.info(f"Loading input data: {[f for f in data]}")
     data_cat, ref_img = io.load_data(data, n_echos=n_echos)
@@ -668,10 +668,19 @@ def tedana_workflow(
     if mixing_file is not None and op.isfile(mixing_file):
         mixing_file = op.abspath(mixing_file)
         # Allow users to re-run on same folder
-        mixing_name = io_generator.get_name("ICA mixing tsv")
-        if mixing_file != mixing_name:
-            shutil.copyfile(mixing_file, mixing_name)
-            shutil.copyfile(mixing_file, op.join(io_generator.out_dir, op.basename(mixing_file)))
+        mixing_name_output = io_generator.get_name("ICA mixing tsv")
+        mixing_file_new_path = op.join(io_generator.out_dir, op.basename(mixing_file))
+        if op.basename(mixing_file) != op.basename(mixing_name_output) and not op.isfile(
+            mixing_file_new_path
+        ):
+            shutil.copyfile(mixing_file, mixing_file_new_path)
+        else:
+            # Add "user_provided" to the mixing file's name if it's identical to the new file name
+            # or if there's already a file in the output directory with the same name
+            shutil.copyfile(
+                mixing_file,
+                op.join(io_generator.out_dir, f"user_provided_{op.basename(mixing_file)}"),
+            )
     elif mixing_file is not None:
         raise OSError("Argument '--mix' must be an existing file.")
 
@@ -841,7 +850,7 @@ def tedana_workflow(
             extra_metrics = ["variance explained", "normalized variance explained", "kappa", "rho"]
             necessary_metrics = sorted(list(set(necessary_metrics + extra_metrics)))
 
-            component_table, _ = metrics.collect.generate_metrics(
+            component_table, mixing = metrics.collect.generate_metrics(
                 data_cat=data_cat,
                 data_optcom=data_optcom,
                 mixing=mixing,
@@ -888,7 +897,7 @@ def tedana_workflow(
                 # regular expressions immediately after initialization,
                 # store and copy this key
                 tmp_external_regressor_config = selector.tree["external_regressor_config"]
-                selector = ComponentSelector(tree)
+                selector = ComponentSelector(tree, out_dir)
                 selector.tree["external_regressor_config"] = tmp_external_regressor_config
 
             RepLGR.disabled = True  # Disable the report to avoid duplicate text
@@ -896,7 +905,6 @@ def tedana_workflow(
         io_generator.overwrite = overwrite  # Re-enable original overwrite behavior
     else:
         LGR.info("Using supplied mixing matrix from ICA")
-        mixing_file = io_generator.get_name("ICA mixing tsv")
         mixing = pd.read_table(mixing_file).values
 
         # selector = ComponentSelector(tree)
@@ -905,7 +913,7 @@ def tedana_workflow(
         extra_metrics = ["variance explained", "normalized variance explained", "kappa", "rho"]
         necessary_metrics = sorted(list(set(necessary_metrics + extra_metrics)))
 
-        component_table, _ = metrics.collect.generate_metrics(
+        component_table, mixing = metrics.collect.generate_metrics(
             data_cat=data_cat,
             data_optcom=data_optcom,
             mixing=mixing,
@@ -926,17 +934,10 @@ def tedana_workflow(
             n_independent_echos=n_independent_echos,
         )
 
-    # TODO The ICA mixing matrix should be written out after it is created
-    #     It is currently being written after component selection is done
-    #     and rewritten if an existing mixing matrix is given as an input
     comp_names = component_table["Component"].values
     mixing_df = pd.DataFrame(data=mixing, columns=comp_names)
-    if not op.exists(io_generator.get_name("ICA mixing tsv")):
-        io_generator.save_file(mixing_df, "ICA mixing tsv")
-    else:  # Make sure the relative path to the supplied mixing matrix is saved in the registry
-        io_generator.registry["ICA mixing tsv"] = op.basename(
-            io_generator.get_name("ICA mixing tsv")
-        )
+    io_generator.save_file(mixing_df, "ICA mixing tsv")
+
     betas_oc = utils.unmask(computefeats2(data_optcom, mixing, mask_denoise), mask_denoise)
     io_generator.save_file(betas_oc, "z-scored ICA components img")
 


Test patch:
diff --git a/tedana/tests/data/nih_five_echo_outputs_verbose.txt b/tedana/tests/data/nih_five_echo_outputs_verbose.txt
index 2d54db9..1976089 100644
--- a/tedana/tests/data/nih_five_echo_outputs_verbose.txt
+++ b/tedana/tests/data/nih_five_echo_outputs_verbose.txt
@@ -121,6 +121,9 @@ figures/sub-01_comp_028.png
 figures/sub-01_comp_029.png
 figures/sub-01_comp_030.png
 figures/sub-01_comp_031.png
+figures/sub-01_comp_032.png
+figures/sub-01_comp_033.png
+figures/sub-01_comp_034.png
 figures/sub-01_rmse_brain.svg
 figures/sub-01_rmse_timeseries.svg
 figures/sub-01_s0_brain.svg
diff --git a/tedana/tests/test_io.py b/tedana/tests/test_io.py
index e204f99..4af9f05 100644
--- a/tedana/tests/test_io.py
+++ b/tedana/tests/test_io.py
@@ -2,15 +2,17 @@
 
 import json
 import os
+from unittest import mock
 
 import nibabel as nib
 import numpy as np
 import pandas as pd
 import pytest
+import requests
 
 from tedana import io as me
 from tedana.tests.test_utils import fnames, tes
-from tedana.tests.utils import get_test_data_path
+from tedana.tests.utils import data_for_testing_info, get_test_data_path
 
 data_dir = get_test_data_path()
 
@@ -318,3 +320,75 @@ def test_custom_encoder():
     encoded = json.dumps(test_data, cls=me.CustomEncoder)
     decoded = json.loads(encoded)
     assert test_data == decoded
+
+
+@mock.patch("tedana.io.requests.get")
+@mock.patch("tedana.io.op.isfile")
+def test_download_json_file_not_found(mock_isfile, mock_requests_get):
+    """Test case when file doesn't exist locally or on figshare"""
+    mock_isfile.return_value = False
+
+    mock_response = mock.Mock()
+    mock_response.raise_for_status = mock.Mock()
+    mock_response.json.return_value = {"files": [{"name": "tree.json", "download_url": "url.com"}]}
+    mock_requests_get.return_value = mock_response
+
+    result = me.download_json("non_existent_tree", "some_dir")
+
+    assert result is None
+    mock_response.raise_for_status.assert_called_once()
+
+
+@mock.patch("tedana.io.requests.get")
+@mock.patch("tedana.io.op.isfile")
+def test_download_json_skips_if_exists(mock_isfile, mock_requests_get):
+    """Test case when file already exists locally"""
+    mock_isfile.return_value = True
+
+    result = me.download_json("my_tree", "some_dir")
+
+    assert result == "some_dir/my_tree.json"
+    mock_requests_get.assert_not_called()
+
+
+@mock.patch("tedana.io.requests.get")
+def test_download_json_doesnt_connect_to_url(mock_requests_get, caplog: pytest.LogCaptureFixture):
+    """Tests that the correct log message appears when URL not connected"""
+    mock_requests_get.side_effect = requests.exceptions.ConnectionError(
+        "Simulated connection error"
+    )
+
+    result = me.download_json("tedana_orig", "./")
+    assert result is None
+    assert "Cannot connect to figshare" in caplog.text
+
+
+@mock.patch("tedana.io.requests.get")
+@mock.patch("tedana.io.op.isfile")
+def test_download_json_file_is_downloaded(mock_isfile, mock_requests_get):
+    """Test json is downloaded if it exists on figshare"""
+    mock_isfile.return_value = False
+
+    metadata_response = mock.Mock()
+    metadata_response.raise_for_status = mock.Mock()
+    metadata_response.json.return_value = {
+        "files": [{"name": "tree.json", "download_url": "url.com"}]
+    }
+
+    download_response = mock.Mock()
+    download_response.raise_for_status = mock.Mock()
+    file_content = {"sample": "data"}
+    download_response.content = json.dumps(file_content).encode("utf-8")
+
+    mock_requests_get.side_effect = [metadata_response, download_response]
+
+    out_dir = data_for_testing_info("path")
+    result = me.download_json("tree.json", out_dir)
+
+    assert result == f"{out_dir}/tree.json"
+    assert os.path.exists(result)
+
+    with open(result, "r") as f:
+        content = json.load(f)
+    assert content == file_content
+    os.remove(result)
diff --git a/tedana/tests/test_metrics.py b/tedana/tests/test_metrics.py
index 0356ca7..4c8ad03 100644
--- a/tedana/tests/test_metrics.py
+++ b/tedana/tests/test_metrics.py
@@ -8,6 +8,7 @@ import pytest
 
 from tedana import io, utils
 from tedana.metrics import collect, dependence, external
+from tedana.metrics._utils import flip_components
 from tedana.tests.test_external_metrics import (
     sample_external_regressor_config,
     sample_external_regressors,
@@ -74,7 +75,7 @@ def test_smoke_generate_metrics(testdata1):
         external_regressors, external_regressor_config, n_vols
     )
 
-    component_table, _ = collect.generate_metrics(
+    component_table, new_mixing = collect.generate_metrics(
         data_cat=testdata1["data_cat"],
         data_optcom=testdata1["data_optcom"],
         mixing=testdata1["mixing"],
@@ -87,6 +88,12 @@ def test_smoke_generate_metrics(testdata1):
         metrics=metrics,
     )
     assert isinstance(component_table, pd.DataFrame)
+    # new_mixing should have flipped signs compared to mixing.
+    # multiplying by "optimal sign" will flip the signs back so it should match
+    assert (
+        np.round(flip_components(new_mixing, signs=component_table["optimal sign"].to_numpy()), 4)
+        == np.round(testdata1["mixing"], 4)
+    ).all()
 
 
 def test_generate_metrics_fails(testdata1):
diff --git a/tedana/tests/utils.py b/tedana/tests/utils.py
index 6147165..0e93f39 100644
--- a/tedana/tests/utils.py
+++ b/tedana/tests/utils.py
@@ -45,15 +45,20 @@ def data_for_testing_info(test_dataset=str):
         three-echo-reclassify
         four-echo
         five-echo
+       To only return the base path without downloading data,
+       you can supply:
+        path
 
     Returns
     -------
     test_data_path : str
        The path to the local directory where the data will be downloaded
+       If "path" is specified, returns the base directory path.
     osf_id : str
        The ID for the OSF file.
        Data download link would be https://osf.io/osf_id/download
        Metadata download link would be https://osf.io/osf_id/metadata/?format=datacite-json
+       If "path" is specified, this value is not returned.
     """
 
     tedana_path = dirname(tedana_cli.__file__)
@@ -79,6 +84,8 @@ def data_for_testing_info(test_dataset=str):
         osf_id = "9c42e"
         makedirs(join(base_data_path, "five-echo"), exist_ok=True)
         makedirs(join(base_data_path, "outputs/five-echo"), exist_ok=True)
+    elif test_dataset == "path":
+        return base_data_path
     else:
         raise ValueError(f"{test_dataset} is not a valid dataset string for data_for_testing_info")