Repository: mindsdb/lightwood

Issue:
:wrench: Implement a quantum mixer :mage: 
# :wrench: Implement a quantum mixer :mage: 

## Introduction

Lightwood is an AutoML library that aims at abstracting common procedures in the Machine Learning usual workflow (e.g. data preparation, feature engineering, model building, model analysis) with an approach that can be fully automated but easily customizable and extended thanks to its flexibility. 

One of the key tenets of Lightwood's philosophy is to engineer features by using _encoders_ that might transform columnar data into a useful descriptor. These are then fed into what is called a _mixer_: the actual ML model tasked with learning to predict the specified target. 

This approach is very flexible. Recently, we've come across the great [Qiskit SDK](https://qiskit.org/) for working with quantum computers, and thought it could be very cool to experiment with _hybrid quantum-classical neural networks_ as mixers! 

## Task

Implement a proof-of-concept `QuantumMixer` class that inherits from our `BaseMixer` class and uses a hybrid quantum neural network as the learning mechanism. We suggest closely following this [Qiskit tutorial](https://qiskit.org/textbook/ch-machine-learning/machine-learning-qiskit-pytorch.html) for an initial implementation.

A good way to measure success is to implement a unit test that uses the quantum mixer in a simple task. For example, you can use [this integration test](https://github.com/mindsdb/lightwood/blob/46ba63adc9037bda45f0c864e72ec1da4029512e/tests/integration/basic/test_airline.py#L8) as reference and opt for the [HDI](https://github.com/mindsdb/lightwood/blob/staging/tests/data/hdi.csv) toy dataset. Be sure to read the [custom mixer tutorial](https://lightwood.io/tutorials/custom_mixer/custom_mixer.html) so that you can force quantum mixer usage in the test.

> Note: mixer accuracy is not relevant at this time, future work can tackle it. Additionally, you can opt for testing with synthetic data if that's easier for you.

The core Lightwood team would love to be of assistance to any contributors interested in tackling this challenge. If at any point you have questions and/or want to discuss relevant details, feel free to post them here or in the community slack.

## Steps :male_detective: :female_detective: 

- Fork the Lightwood repository, checkout the `staging` branch and from it create a new one.
- Implement the `QuantumMixer` class, using all resources suggested above. Any files should be added inside the `lightwood/encoder/quantum` directory.
- Implement a unit test where your mixer is successfully used with a sample dataset.
- Make the PR and address any comments that reviewers might make.

## Additional rewards :1st_place_medal: 

This challenge encompasses mixer research. As such, an accepted PR brings :one::zero: points for entry into the draw for a :computer:  Deep Learning Laptop powered by the NVIDIA RTX 3080 Max-Q GPU or other swag :shirt: :bear: .  For more info check out https://mindsdb.com/hacktoberfest/


Discussion:
Hi saw that there alr is a pr, but hope to work on this qiskit implementation too, I a have slightly different implementation that kinda builds on default net, will it be considered duplicated effort? 
> Hi saw that there alr is a pr, but hope to work on this qiskit implementation too, I a have slightly different implementation that kinda builds on default net, will it be considered duplicated effort?

Hey, I am curious about you implementation too so I suppose you could open a PR as well. We could even join our efforts and pick the best parts of both our solutions. I suppose splitting the points would not be a problem for them at all.

Do you also treat one quantum circuit as a ℝ ->ℝ function, using them as if it were just like any nonlinearity?

Hi @ongspxm, yes I think they sound distinct enough that we can accept both with full points!

Resolving patch:
diff --git a/lightwood/mixer/__init__.py b/lightwood/mixer/__init__.py
--- a/lightwood/mixer/__init__.py
+++ b/lightwood/mixer/__init__.py
@@ -6,5 +6,9 @@
 from lightwood.mixer.sktime import SkTime
 from lightwood.mixer.regression import Regression
 
+try:
+    from lightwood.mixer.qclassic import QClassic
+except Exception:
+    QClassic = None
 
-__all__ = ['BaseMixer', 'Neural', 'LightGBM', 'LightGBMArray', 'Unit', 'Regression', 'SkTime']
+__all__ = ['BaseMixer', 'Neural', 'LightGBM', 'LightGBMArray', 'Unit', 'Regression', 'SkTime', 'QClassic']
diff --git a/lightwood/mixer/helpers/qclassic_net.py b/lightwood/mixer/helpers/qclassic_net.py
new file mode 100644
--- /dev/null
+++ b/lightwood/mixer/helpers/qclassic_net.py
@@ -0,0 +1,125 @@
+import torch
+import qiskit
+import numpy as np
+
+from lightwood.mixer.helpers.default_net import DefaultNet
+from lightwood.helpers.torch import LightwoodAutocast
+
+
+class QuantumCircuit:
+    """
+    This class provides a simple interface for interaction
+    with the quantum circuit
+    """
+
+    def __init__(self, n_qubits, backend, shots):
+        # --- Circuit definition ---
+        self._circuit = qiskit.QuantumCircuit(n_qubits)
+
+        all_qubits = [i for i in range(n_qubits)]
+        self.theta = qiskit.circuit.Parameter('theta')
+
+        self._circuit.h(all_qubits)
+        self._circuit.barrier()
+        self._circuit.ry(self.theta, all_qubits)
+
+        self._circuit.measure_all()
+        # ---------------------------
+
+        self.backend = backend
+        self.shots = shots
+
+    def run(self, thetas):
+        job = self.backend.run([self._circuit.bind_parameters({self.theta: t})
+                                for t in thetas], shots=self.shots)
+        results = job.result().get_counts()
+
+        final = []
+        for result in results:
+            counts = np.array(list(result.values()))
+            states = np.array(list(result.keys())).astype(float)
+
+            # Compute probabilities for each state
+            probabilities = counts / self.shots
+
+            # Get state expectation
+            expectation = np.sum(states * probabilities)
+            final.append(expectation)
+        return np.array(final)
+
+
+class HybridSingleFunction(torch.autograd.Function):
+    """ Hybrid quantum - classical function definition """
+
+    @staticmethod
+    def forward(ctx, input, quantum_circuit, shift):
+        """ Forward pass computation """
+        ctx.shift = shift
+        ctx.quantum_circuit = quantum_circuit
+
+        expectation_z = ctx.quantum_circuit.run(input.tolist())
+        result = torch.tensor(expectation_z)
+        ctx.save_for_backward(input, result)
+
+        return result
+
+    @staticmethod
+    def backward(ctx, grad_output):
+        """ Backward pass computation """
+        input, expectation_z = ctx.saved_tensors
+        input_list = np.array(input.tolist())
+
+        shift_right = input_list + np.ones(input_list.shape) * ctx.shift
+        shift_left = input_list - np.ones(input_list.shape) * ctx.shift
+
+        expectation_left = ctx.quantum_circuit.run(shift_left)
+        expectation_right = ctx.quantum_circuit.run(shift_right)
+        gradients = torch.tensor([expectation_right]) - torch.tensor([expectation_left])
+        gradients = np.array(gradients).T
+        return torch.tensor(gradients).float() * grad_output.float(), None, None
+
+
+class HybridSingle(torch.nn.Module):
+    """ Hybrid quantum - classical layer definition """
+
+    def __init__(self, backend, shots, shift):
+        super(HybridSingle, self).__init__()
+        self.quantum_circuit = QuantumCircuit(1, backend, shots)
+        self.shift = shift
+
+    def forward(self, input):
+        return HybridSingleFunction.apply(input, self.quantum_circuit, self.shift)
+
+
+class QClassicNet(DefaultNet):
+    """
+    DefaultNet variant that uses qiskit to add a final quantum layer
+    """
+
+    def __init__(self,
+                 input_size: int = None,
+                 output_size: int = None,
+                 shape: list = None,
+                 max_params: int = 3e7,
+                 num_hidden: int = 1,
+                 dropout: float = 0) -> None:
+        hidden_size = max([input_size * 2, output_size * 2, 400])
+        super().__init__(input_size=input_size,
+                         output_size=hidden_size,
+                         shape=shape,
+                         max_params=max_params,
+                         num_hidden=num_hidden,
+                         dropout=dropout)
+        self.fc = torch.nn.Linear(hidden_size, output_size)
+        self.hybrid = HybridSingle(qiskit.Aer.get_backend('aer_simulator'), 100, np.pi / 2)
+
+    def to(self, device=None, available_devices=None):
+        return super().to(device)
+
+    def forward(self, input):
+        with LightwoodAutocast():
+            if len(input.shape) == 1:
+                input = input.unsqueeze(0)
+            classical_output = self.fc(self.net(input))
+            full_output = torch.stack([self.hybrid(i) for i in classical_output])
+        return full_output.float()
diff --git a/lightwood/mixer/qclassic.py b/lightwood/mixer/qclassic.py
new file mode 100644
--- /dev/null
+++ b/lightwood/mixer/qclassic.py
@@ -0,0 +1,21 @@
+from typing import Dict, List
+
+from lightwood.encoder.base import BaseEncoder
+from lightwood.mixer.neural import Neural
+from lightwood.mixer.helpers.qclassic_net import QClassicNet
+from lightwood.api.types import TimeseriesSettings
+
+
+class QClassic(Neural):
+    # wrapper class to be combined with Neural class when performance stabilizes
+    def __init__(
+            self, stop_after: int, target: str, dtype_dict: Dict[str, str],
+            input_cols: List[str],
+            timeseries_settings: TimeseriesSettings, target_encoder: BaseEncoder, net: str, fit_on_dev: bool,
+            search_hyperparameters: bool):
+        super().__init__(stop_after, target, dtype_dict,
+                         input_cols, timeseries_settings, target_encoder,
+                         net, fit_on_dev, search_hyperparameters)
+
+        quantum_nets = {"QClassic": QClassicNet}
+        self.net_class = quantum_nets.get(net, QClassicNet)


Test patch:
diff --git a/tests/integration/basic/test_qclassic.py b/tests/integration/basic/test_qclassic.py
new file mode 100644
--- /dev/null
+++ b/tests/integration/basic/test_qclassic.py
@@ -0,0 +1,45 @@
+import unittest
+import pandas as pd
+from sklearn.metrics import accuracy_score
+
+from lightwood.api.high_level import ProblemDefinition, json_ai_from_problem
+from lightwood.api.high_level import code_from_json_ai, predictor_from_code
+
+from lightwood.mixer import QClassic
+
+
+class TestBasic(unittest.TestCase):
+    def test_0_predict_file_flow(self):
+        if QClassic is None:
+            print('Skipping this test since the system for the quantum are not installed')
+            return
+
+        df = pd.read_csv('tests/data/hdi.csv')[:500]
+        target = 'Development Index'
+
+        pdef = ProblemDefinition.from_dict({'target': target, 'time_aim': 80})
+        json_ai = json_ai_from_problem(df, problem_definition=pdef)
+
+        neural_args = json_ai.outputs[target].mixers[0]['args']
+        neural_args["target_encoder"] = "$encoders[self.target]"
+        neural_args["target"] = "$target"
+        neural_args["dtype_dict"] = "$dtype_dict"
+        neural_args["input_cols"] = "$input_cols"
+        neural_args["timeseries_settings"] = "$problem_definition.timeseries_settings"
+        neural_args["net"] = '"DefaultNet"'
+
+        json_ai.outputs[target].mixers = [{
+            'module': 'lightwood.mixer.QClassic',
+            'args': neural_args
+        }]
+
+        code = code_from_json_ai(json_ai)
+        predictor = predictor_from_code(code)
+
+        predictor.learn(df)
+        predictions = predictor.predict(df)
+        return
+
+        # sanity checks
+        self.assertTrue(accuracy_score(df[target], predictions['prediction']) > 0.5)
+        self.assertTrue(all([0 <= p <= 1 for p in predictions['confidence']]))