I am doing optimization process of my analog circuit design, where i need to optimally tune the input circuit design variables (13 variables) to attain the output target objective performance metrics (7 metrics). Here my circuit topology/circuit netlist, is fixed, I only need to tune the 13 input variables to reach the target objective metrics.
With the below custom 'circuitEnvironment' calss, I need you to provide very best approach to implement the following and utilize the given class 'circuitEnvironment' output variables/parameters which are all necessary for implementing RL-GNN optimistically,
1. Need to Define best suitable GNN model.
define a best suitable GNN model with all its assiciated functions. The suggested GNN model is need to be capable of taking, 'node_features_tensor', 'edge_embeddings(edge feature tensor)', 'adjacency_matrix_tensor' all three as inputs state and predict the circuit’s performance metrics. Because, I derived my three graph features (node features, edge features, and adjacency matrix) from undirected bipartite miltigraph with fixed topology, where i have parallel edges between two same nodes. So, the edge features along with the node features are necessary for the model to achieve the required objectives. I need you to provide all the necessary complete structures to be include with in the GNN model and the structure need to be properly interface with the RL algorithm to achieve my required objectives.
2. Set up Reinforcement Learning with Proximal Policy Optimization(PPO) as an agent:
For the RL part, need to be working with the environment and possibly the predictions from your GNN. 'node_features_tensor, edge_embeddings, adjacency_matrix_tensor, reward, done, previous_metrics = env.step(self, raw_actions, previous_metrics)'. I need you to suggest all necessary complete structures to be include with in the PPO agent in RL algorithm for my implementation.
3. Training Loop:
Finally, I need a training loop where the RL and PPO agent interacts with the environment (modifying the circuit’s parameters to meet performance targets) and possibly incorporates the GNN’s predictions. initialize the environment with 'node_features_tensor, edge_embeddings, adjacency_matrix_tensor, performance_metrics = env.reset()'. Provide me the complete structures to be include with in the training loop to achieve best performance.

Below code is My exixting implementation of class 'CircuitEnvironment' for RL PPO policy interaction to receive reward and state values 'node_features_tensor', 'edge_embeddings(edge feature tensor)', 'adjacency_matrix_tensor' from the environment.

import numpy as np
import torch
import torch.nn as nn
import torch.optim as optim
import torch.distributions
from torch.utils.data import TensorDataset, DataLoader
from Simulator_Env import CircuitSimulator
from Netlist_Content import get_netlist_content
from NetToGraph import netlist_to_graph

class CircuitEnvironment:
    def __init__(self, server_address, username, password, bounds_low, bounds_high, target_metrics, netlist_content):
        self.server_address = server_address
        self.username = username
        self.password = password
        self.bounds_low = bounds_low
        self.bounds_high = bounds_high
        self.target_metrics = target_metrics
        self.previous_transistor_regions = None 
        self.simulator = None
        
        self.netlist_content = netlist_content  # Fetch netlist content upon environment initialization
        # Initialize tensors as None 
        self.node_features_tensor = None
        self.edge_embeddings = None
        self.adjacency_matrix = None

    def connect_to_simulator(self):
        self.simulator = CircuitSimulator(self.server_address, self.username, self.password)
        self.simulator.connect()

    def disconnect_from_simulator(self):
        if self.simulator:
            self.simulator.close()
            self.simulator = None      

    def reset(self):
        # Connect to the simulator
        self.connect_to_simulator()        
        # Initialize parameters within bounds
        initial_parameters = np.random.uniform(self.bounds_low, self.bounds_high)
        
        # Run simulation with initial parameters
        transistor_regions, saturation_condition, performance_metrics = self.simulator.run_simulation(initial_parameters)
        # Process netlist to generate model inputs
        node_features, node_features_tensor, edge_features, edge_embeddings, adjacency_matrix, adjacency_matrix_tensor, degree_matrix = netlist_to_graph(self.netlist_content, initial_parameters)
        
        return node_features_tensor, edge_embeddings, adjacency_matrix_tensor, performance_metrics

    def step(self, raw_actions, previous_metrics):
        # Ensure raw_actions is a 1D array
        raw_actions = np.squeeze(raw_actions)                     
        # Run simulation with new parameters
        transistor_regions, saturation_condition, performance_metrics = self.simulator.run_simulation(raw_actions)
        # Process netlist to generate model inputs
        node_features, node_features_tensor, edge_features, edge_embeddings, adjacency_matrix, adjacency_matrix_tensor, degree_matrix = netlist_to_graph(self.netlist_content, raw_actions)
        
        # Calculate reward based on performance metrics and transistor_regions
        reward = self.calculate_reward(performance_metrics, previous_metrics, transistor_regions, self.previous_transistor_regions)
        # Check if the episode is done
        done = self.check_done(performance_metrics, encoded_saturation_condition)
        
        return node_features_tensor, edge_embeddings, adjacency_matrix_tensor, reward, done, previous_metrics

    def is_performance_metrics_in_target(self, metrics):
        return np.all(metrics >= self.PERFORMANCE_METRICS_TARGET_LOW) and np.all(metrics <= self.PERFORMANCE_METRICS_TARGET_HIGH)      
    
    def calculate_reward(self, c_metrics, p_metrics, transistor_regions, previous_transistor_regions):

        return reward
       
    def check_done(self, performance_metrics, encoded_saturation_condition):

        return True
    
# Define the values for bounds, and target metrics
bounds_low = np.array([0.18e-6, 0.18e-6, 0.18e-6, 0.18e-6, 0.18e-6, 0.5e-6, 0.5e-6, 0.5e-6, 0.5e-6, 0.5e-6, 15e-6, 0.1e-12, 0.8])
bounds_high = np.array([0.2e-6, 0.2e-6, 0.2e-6, 0.2e-6, 0.2e-6, 50e-6, 50e-6, 50e-6, 50e-6, 50e-6, 30e-6, 10e-12, 1.4])

netlist_content = get_netlist_content()

# Create the environment
env = CircuitEnvironment(server_address, username, password, bounds_low, bounds_high, target_metrics, netlist_content)

node_features_tensor, edge_embeddings, adjacency_matrix_tensor, performance_metrics = env.reset()