Repository: modin-project/modin
Pull request: FIX-#4044: Update module layout with the ModinDataframe
Signed-off-by: Rehan Durrani <rehan@ponder.io>

<!--
Thank you for your contribution!
Please review the contributing docs: https://modin.readthedocs.io/en/latest/developer/contributing.html
if you have questions about contributing.
-->

## What do these changes do?
This PR updates the System Architecture documentation page's module layout with the `ModinDataframe` to reflect the changes from #2656.
<!-- Please give a short brief about these changes. -->

- [x] commit message follows format outlined [here](https://modin.readthedocs.io/en/latest/developer/contributing.html#commit-message-formatting)
- [x] passes `flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] passes `black --check modin/ asv_bench/benchmarks scripts/doc_checker.py`
- [x] signed commit with `git commit -s` <!-- you can amend your commit with a signature via `git commit -amend -s` -->
- [x] Resolves #4044 ? <!-- issue must be created for each patch -->
- [x] tests added and passing
- [x] module layout described at `docs/development/architecture.rst` is up-to-date <!-- if you have added, renamed or removed files or directories please update the documentation accordingly -->


Problem statement:
Fix documentation after algebra refactor
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. What kind of performance improvements would you like to see with this new API?
The module layout in the system architecture doc is now inaccurate thanks to both the major refactor from last year, and the refactor which introduces the `ModinDataframe`.

Hints from issue discussion:

Commit message: FIX-#4044: Update module layout with the ModinDataframe

Signed-off-by: Rehan Durrani <rehan@ponder.io>

Patch to review:
diff --git a/docs/flow/modin/core/dataframe/base/dataframe.rst b/docs/flow/modin/core/dataframe/base/dataframe.rst
new file mode 100644
index 00000000000..1303604267f
--- /dev/null
+++ b/docs/flow/modin/core/dataframe/base/dataframe.rst
@@ -0,0 +1,22 @@
+ModinDataframe
+"""""""""""""""
+
+The :py:class:`~modin.core.dataframe.base.dataframe.dataframe.ModinDataframe` is the parent class for all dataframes - regardless of what storage format they are backed by. Its purpose is to define the algebra operators that must be exposed by a dataframe.
+
+This class exposes the dataframe algebra and is meant to be subclassed by all dataframe implementations.
+Descendants of this class implement the algebra, and act as the intermediate level
+between the query compiler and the underlying exection details (e.g. the conforming partition manager). The class provides
+a significantly reduced set of operations that can be composed to form any pandas query.
+
+
+The ``PandasDataframe`` is an example of a descendant of this class. It currently has implementations for some of the operators
+exposed in this class, and is currently being refactored to include implementations for all of the algebra operators. Please
+refer to the :doc: `PandasDataframe documentation<flow/modin/core/dataframe/pandas/dataframe.rst>` for more information.
+
+The ``ModinDataframe`` is independent of implementation specific details such as partitioning, storage format, or execution engine.
+
+Public API
+----------
+
+.. autoclass:: modin.core.dataframe.base.dataframe.dataframe.ModinDataframe
+  :members:
diff --git a/docs/flow/modin/core/dataframe/base/index.rst b/docs/flow/modin/core/dataframe/base/index.rst
index a4cb1ed13e6..20c180615c0 100644
--- a/docs/flow/modin/core/dataframe/base/index.rst
+++ b/docs/flow/modin/core/dataframe/base/index.rst
@@ -1,28 +1,25 @@
 Purpose
 =======
 
-``BaseDataframe`` serves the purpose of describing and defining the :doc:`Core Dataframe Algebra </flow/modin/core/dataframe/algebra>`.
+The ``ModinDataframe`` serves the purpose of describing and defining the :doc:`Core Dataframe Algebra </flow/modin/core/dataframe/algebra>`.
 
-It is the core construction element which serves as the client for :doc:`Modin Query Compiler</flow/modin/core/storage_formats/base/query_compiler>` and which implementations are actually executing the queries from the compiler by invoking functions over partition(s).
+It is the core construction element and serves as the client for the :doc:`Modin Query Compiler</flow/modin/core/storage_formats/base/query_compiler>`. Descendants that offer implementations execute the queries from the compiler by invoking functions over partitions via a partition manager.
 
-To execute the queries, a typical implementation also itroduces partitions and
-partition manager, interfaces for which we might consider standardising in the future.
-For now they're totally implementation-specific.
+The partitions and partition manager interfaces are currently implementation-specific, but may
+be standardized in the future.
 
-Base dataframe and axis partitions are the interfaces that must be implemented by any :doc:`execution backend</flow/modin/core/execution/dispatching>` that wants to be plugged in Modin.
+The ModinDataframe and axis partitions are the interfaces that must be implemented by any :doc:`execution backend</flow/modin/core/execution/dispatching>` in order for it to be plugged in to Modin.
 These classes are mostly abstract, however very simple and generic enough methods like
 :py:meth:`~modin.core.dataframe.base.partitioning.BaseDataframeAxisPartition.force_materialization` can be implemented at the base level because for now we do not expect them to differ in any implementation.
 
-Modin BaseDataframe Interface
+ModinDataframe Interface
 =============================
 
+* :doc:`ModinDataframe <dataframe>` is an abstract class which represents the algebra operators a dataframe must expose.
 * :doc:`BaseDataframeAxisPartition <partitioning/axis_partition>` is an abstract class, representing a joined group of partitions along some axis (either rows or labels).
 
-.. note::
-    Common interfaces for most of the Modin Dataframe objects are not defined yet. Currently, all of the implementations
-    inherit :doc:`Dataframe implementation for pandas storage format</flow/modin/core/dataframe/pandas/index>`.
-
 .. toctree::
     :hidden:
 
+    dataframe
     partitioning/axis_partition
diff --git a/docs/flow/modin/core/dataframe/pandas/dataframe.rst b/docs/flow/modin/core/dataframe/pandas/dataframe.rst
index 70634b774d3..8bac2a1244e 100644
--- a/docs/flow/modin/core/dataframe/pandas/dataframe.rst
+++ b/docs/flow/modin/core/dataframe/pandas/dataframe.rst
@@ -1,7 +1,7 @@
 PandasDataframe
 """""""""""""""
 
-:py:class:`~modin.core.dataframe.pandas.dataframe.dataframe.PandasDataframe` should be a direct descendant of ``BaseDataframe`` which is being factored right now. Its purpose is to implement the abstract interfaces for usage with all ``pandas``-based :doc:`storage formats<flow/modin/core/storage_formats/index.html>`.
+:py:class:`~modin.core.dataframe.pandas.dataframe.dataframe.PandasDataframe` is a direct descendant of ``ModinDataframe``. Its purpose is to implement the abstract interfaces for usage with all ``pandas``-based :doc:`storage formats<flow/modin/core/storage_formats/index.html>`.
 :py:class:`~modin.core.dataframe.pandas.dataframe.dataframe.PandasDataframe` could be inherited and augmented further by any specific implementation which needs it to take special care of some behavior or to improve performance for certain execution engine.
 
 The class serves as the intermediate level
diff --git a/docs/flow/modin/core/dataframe/pandas/index.rst b/docs/flow/modin/core/dataframe/pandas/index.rst
index 71d76c3e8b5..e49b04ff604 100644
--- a/docs/flow/modin/core/dataframe/pandas/index.rst
+++ b/docs/flow/modin/core/dataframe/pandas/index.rst
@@ -18,4 +18,4 @@ as it implements everything an execution backend needs to be fully conformant to
     dataframe
     partitioning/partition
     partitioning/axis_partition
-    partitioning/partition_manager
\ No newline at end of file
+    partitioning/partition_manager


Code review comments:
diff_hunk: @@ -1,28 +1,25 @@
 Purpose
 =======
 
-``BaseDataframe`` serves the purpose of describing and defining the :doc:`Core Dataframe Algebra </flow/modin/core/dataframe/algebra>`.
+The ``ModinDataframe`` serves the purpose of describing and defining the :doc:`Core Dataframe Algebra </flow/modin/core/dataframe/algebra>`.
 
-It is the core construction element which serves as the client for :doc:`Modin Query Compiler</flow/modin/core/storage_formats/base/query_compiler>` and which implementations are actually executing the queries from the compiler by invoking functions over partition(s).
+It is the core construction element and serves as the client for the :doc:`Modin Query Compiler</flow/modin/core/storage_formats/base/query_compiler>`. Descendants that offer implementations execute the queries from the compiler by invoking functions over partitions via a partition manager.
 
-To execute the queries, a typical implementation also itroduces partitions and
-partition manager, interfaces for which we might consider standardising in the future.
-For now they're totally implementation-specific.
+The partitions and partition manager interfaces are currently implementation-specific, but may
+be standardized in the future.
 
-Base dataframe and axis partitions are the interfaces that must be implemented by any :doc:`execution backend</flow/modin/core/execution/dispatching>` that wants to be plugged in Modin.
+The ModinDataframe and axis partitions are the interfaces that must be implemented by any :doc:`execution backend</flow/modin/core/execution/dispatching>` in order for it to be plugged in to Modin.
path: docs/flow/modin/core/dataframe/base/index.rst
text: @user1:
```suggestion
The :py:class:`~modin.core.dataframe.base.dataframe.dataframe.ModinDataframe` and axis partitions are the interfaces that must be implemented by any :doc:`execution backend</flow/modin/core/execution/dispatching>` in order for it to be plugged in to Modin.
```

diff_hunk: @@ -1,28 +1,25 @@
 Purpose
 =======
 
-``BaseDataframe`` serves the purpose of describing and defining the :doc:`Core Dataframe Algebra </flow/modin/core/dataframe/algebra>`.
+The ``ModinDataframe`` serves the purpose of describing and defining the :doc:`Core Dataframe Algebra </flow/modin/core/dataframe/algebra>`.
 
-It is the core construction element which serves as the client for :doc:`Modin Query Compiler</flow/modin/core/storage_formats/base/query_compiler>` and which implementations are actually executing the queries from the compiler by invoking functions over partition(s).
+It is the core construction element and serves as the client for the :doc:`Modin Query Compiler</flow/modin/core/storage_formats/base/query_compiler>`. Descendants that offer implementations execute the queries from the compiler by invoking functions over partitions via a partition manager.
 
-To execute the queries, a typical implementation also itroduces partitions and
-partition manager, interfaces for which we might consider standardising in the future.
-For now they're totally implementation-specific.
+The partitions and partition manager interfaces are currently implementation-specific, but may
+be standardized in the future.
 
-Base dataframe and axis partitions are the interfaces that must be implemented by any :doc:`execution backend</flow/modin/core/execution/dispatching>` that wants to be plugged in Modin.
+The ModinDataframe and axis partitions are the interfaces that must be implemented by any :doc:`execution backend</flow/modin/core/execution/dispatching>` in order for it to be plugged in to Modin.
 These classes are mostly abstract, however very simple and generic enough methods like
 :py:meth:`~modin.core.dataframe.base.partitioning.BaseDataframeAxisPartition.force_materialization` can be implemented at the base level because for now we do not expect them to differ in any implementation.
 
-Modin BaseDataframe Interface
+ModinDataframe Interface
 =============================
path: docs/flow/modin/core/dataframe/base/index.rst
text: @user1:
```suggestion
ModinDataframe Interface
========================
```

diff_hunk: @@ -1,7 +1,7 @@
 PandasDataframe
 """""""""""""""
 
-:py:class:`~modin.core.dataframe.pandas.dataframe.dataframe.PandasDataframe` should be a direct descendant of ``BaseDataframe`` which is being factored right now. Its purpose is to implement the abstract interfaces for usage with all ``pandas``-based :doc:`storage formats<flow/modin/core/storage_formats/index.html>`.
+:py:class:`~modin.core.dataframe.pandas.dataframe.dataframe.PandasDataframe` is a direct descendant of ``ModinDataframe``. Its purpose is to implement the abstract interfaces for usage with all ``pandas``-based :doc:`storage formats<flow/modin/core/storage_formats/index.html>`.
path: docs/flow/modin/core/dataframe/pandas/dataframe.rst
text: @user1:
```suggestion
:py:class:`~modin.core.dataframe.pandas.dataframe.dataframe.PandasDataframe` is a direct descendant of :py:class:`~modin.core.dataframe.base.dataframe.dataframe.ModinDataframe`. Its purpose is to implement the abstract interfaces for usage with all ``pandas``-based :doc:`storage formats<flow/modin/core/storage_formats/index.html>`.
```