\documentclass[12pt,a4paper]{article}

% --------------------------------------------------
% Packages for enhanced formatting and typography
% --------------------------------------------------
\usepackage{amsmath, amssymb, amsfonts}
\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{listings}
\usepackage{float}
\usepackage{geometry}      % For margin settings
\usepackage{microtype}     % For better typography
\usepackage{lmodern}       % For enhanced font
\usepackage{fancyhdr}      % For custom headers and footers
\usepackage{titlesec}      % For customizing section titles
\usepackage{xcolor}        % For color customization
\usepackage{tcolorbox}     % For creating colored boxes around code

% --------------------------------------------------
% Geometry settings for margins
% --------------------------------------------------
\geometry{
    left=1in,
    right=1in,
    top=1in,
    bottom=1in,
    headheight=15pt
}

% --------------------------------------------------
% Header and Footer settings
% --------------------------------------------------
\pagestyle{fancy}
\fancyhf{}
\fancyhead[L]{Solving ODEs with Neural Networks}
\fancyhead[R]{\thepage}
\fancyfoot[C]{}

% --------------------------------------------------
% Section title formatting
% --------------------------------------------------
\titleformat{\section}
  {\Large\bfseries\color{blue}} % Format
  {\thesection}{1em}{}          % Label

\titleformat{\subsection}
  {\large\bfseries\color{cyan}}
  {\thesubsection}{1em}{}

% --------------------------------------------------
% Listings settings for Python code
% --------------------------------------------------
\lstset{
    language=Python,
    basicstyle=\ttfamily\footnotesize,  % Slightly smaller font
    keywordstyle=\color{blue}\bfseries,
    stringstyle=\color{orange},
    commentstyle=\color{green!50!black}\itshape,
    numbers=left,
    numberstyle=\tiny\color{gray},
    stepnumber=1,
    numbersep=5pt,
    backgroundcolor=\color{gray!10},
    frame=single,
    breaklines=true,
    breakatwhitespace=false,
    showspaces=false,
    showstringspaces=false,
    captionpos=b
}

% --------------------------------------------------
% Tcolorbox settings for code blocks
% --------------------------------------------------
\newtcolorbox{codebox}{
    colback=gray!5,                    % Light gray background
    colframe=gray!75!black,           % Gray frame
    boxrule=0.5pt,                     % Thin border
    arc=4pt,                           % Rounded corners
    auto outer arc,
    left=5pt,                          % Horizontal padding
    right=5pt,
    top=5pt,                           % Vertical padding
    bottom=5pt,
    enhanced,
    breakable,                         % Allow box to break across pages
    fontupper=\footnotesize\ttfamily,  % Smaller font inside box
    listing only,                      % Only display listing
    listing options={language=Python},
    before skip=10pt,                  % Space before the box
    after skip=10pt,                   % Space after the box
    title style={font=\bfseries\small\color{black}}, % Title font
    title=Code Snippet                  % Default title (can be overridden)
}

% --------------------------------------------------
% Title Information
% --------------------------------------------------
\title{\textbf{Solving Ordinary Differential Equations using Neural Networks}}
\author{[Your Name]}
\date{}

% --------------------------------------------------
% Document Start
% --------------------------------------------------
\begin{document}

\maketitle

\tableofcontents

\newpage  % Start Introduction on a new page

\section{Introduction}
Ordinary Differential Equations (ODEs) are fundamental in modeling various physical and biological phenomena. Traditional numerical methods for solving ODEs, while widely used, can be computationally expensive. Recently, neural networks have gained interest for solving ODEs due to their high accuracy and fast convergence.

This assignment explores the application of neural networks to solve a specific second-order ODE. We will discuss the methodology, implement the solution using a neural network, and compare the results with the analytical solution.

\subsection{Background}
Neural networks, inspired by biological nervous systems, consist of interconnected nodes or "neurons" that process and transmit information. In the context of solving ODEs, neural networks act as universal function approximators, capable of learning complex relationships between variables.

The basic structure of a neural network includes an input layer, one or more hidden layers, and an output layer. Each neuron in these layers applies a nonlinear activation function to the weighted sum of its inputs. Common activation functions include the sigmoid function:

\begin{equation}
\sigma(t) = \frac{1}{1 + e^{-t}}, \quad \sigma : \mathbb{R} \rightarrow \mathbb{R}
\end{equation}

and the hyperbolic tangent (tanh) function:

\begin{equation}
\tanh(t) = \frac{e^{t} - e^{-t}}{e^{t} + e^{-t}}, \quad \tanh : \mathbb{R} \rightarrow \mathbb{R}
\end{equation}

\section{Methodology}

\subsection{Problem Formulation}
We consider the following second-order ODE:

\begin{equation}
\frac{d^2y}{dt^2} + y(t) = 0, \quad t > 0
\end{equation}

with initial conditions:

\begin{equation}
y(0) = 1, \quad \frac{dy}{dt}(0) = 0
\end{equation}

The exact analytical solution for this equation is:

\begin{equation}
y(t) = \cos(t)
\end{equation}

\subsection{Trial Solution Method}
To solve this ODE using a neural network, we construct a trial solution that satisfies both the differential equation and its initial conditions. Our trial solution takes the form:

\begin{equation}
y_{trial}(t) = 1 + t^2 \cdot N(t,\vec{p})
\end{equation}

where \( N(t,\vec{p}) \) represents the output of the neural network with parameters \( \vec{p} \). This form ensures that the initial conditions are satisfied regardless of the neural network's output.

\subsection{Neural Network Architecture}
We use a feedforward neural network with the following architecture:
\begin{itemize}
    \item \textbf{Input layer}: 1 neuron (for the independent variable \( t \))
    \item \textbf{Hidden layers}: 2 layers with 200 and 100 neurons respectively
    \item \textbf{Output layer}: 1 neuron (for the dependent variable \( y \))
\end{itemize}

The network uses the sigmoid activation function in the hidden layers.

\section{Implementation}

\subsection{Code Structure}
The implementation is done in Python using the Autograd library for automatic differentiation. Here are the key components of our code:

\subsubsection{Activation Function}
\begin{codebox}
\lstset{title=Sigmoid Activation Function}
\begin{lstlisting}
def sigmoid(z):
    return 1 / (1 + np.exp(-z))
\end{lstlisting}
\end{codebox}

\subsubsection{Neural Network Structure}
\begin{codebox}
\lstset{title=Feedforward Neural Network}
\begin{lstlisting}
def neural_network(parameters, x_in):
    n_hidden = len(parameters) - 1
    num_values = len(x_in)
    x_in = x_in.reshape(-1, num_values)
    x_prev = x_in

    for i in range(n_hidden):
        w_hidden = parameters[i]
        x_prev = np.vstack([np.ones((1, num_values)), x_prev])
        z_hidden = np.matmul(w_hidden, x_prev)
        x_hidden = sigmoid(z_hidden)
        x_prev = x_hidden

    w_output = parameters[-1]
    x_prev = np.vstack([np.ones((1, num_values)), x_prev])
    z_output = np.matmul(w_output, x_prev)
    x_output = z_output

    return x_output
\end{lstlisting}
\end{codebox}

\subsubsection{ODE Solver}
\begin{codebox}
\lstset{title=Neural Network ODE Solver}
\begin{lstlisting}
def neural_network_solve_ode(x_in, num_neurons, num_iterations, learning_rate):
    n_hidden = len(num_neurons)
    parameters = [None] * (n_hidden + 1)
    parameters[0] = npr.randn(num_neurons[0], 2)
    for j in range(1, n_hidden):
        parameters[j] = npr.randn(num_neurons[j], num_neurons[j - 1] + 1)
    parameters[-1] = npr.randn(1, num_neurons[-1] + 1)

    print('Initial cost: %g' % cost_function(parameters, x_in))

    cost_function_grad = grad(cost_function, 0)
    for i in range(num_iterations):
        cost_grad = cost_function_grad(parameters, x_in)
        for j in range(n_hidden + 1):
            parameters[j] -= learning_rate * cost_grad[j]

    print('Final cost: %g' % cost_function(parameters, x_in))

    return parameters
\end{lstlisting}
\end{codebox}

\subsection{Main Execution}

\begin{codebox}
\lstset{title=Main Execution Block}
\begin{lstlisting}
if __name__ == "__main__":
    # Set random seed for reproducibility
    npr.seed(4155)
    
    # Define input range and network parameters
    nx = 10
    x = np.linspace(0, 1, nx)
    num_hidden_neurons = [200, 100]
    num_iterations = 1000
    learning_rate = 1e-3

    # Training the Neural Network
    t1_start = process_time()  # Start timing for training
    P = neural_network_solve_ode(x, num_hidden_neurons, num_iterations, learning_rate)
    t1_stop = process_time()  # Stop timing for training

    # DNN Approximation
    g_dnn_ag = trial_solution(x, P)
    g_analytical = exact_solution(x)

    # Implicit Euler for comparison
    tnum_start = process_time()  # Start timing for Euler method
    tspan = np.array([0, 1])
    y0 = [1, 0]
    t_euler, s_euler = implicit_euler(system_deriv, tspan, y0, num_steps=nx)
    tnum_stop = process_time()  # Stop timing for Euler method

    # Calculate the maximum absolute difference
    max_diff = np.max(np.abs(g_dnn_ag - g_analytical))
    mse = np.mean((g_dnn_ag - g_analytical)**2)

    # Display results
    print(f"The max absolute difference between DNN and analytical solution: {max_diff}")
    print(f"Mean squared error: {mse}")
    print(f"Training time: {t1_stop - t1_start} seconds")

    # Plot the results
    plt.figure(figsize=(10, 6))
    plt.plot(x, g_analytical, 'o', label='Exact Solution')
    plt.plot(x, g_dnn_ag[0], label='Neural Network Approximation')
    plt.plot(t_euler, s_euler[:, 0], label='Implicit Euler Approximation')
    plt.legend()
    plt.xlabel('t')
    plt.ylabel('y(t)')
    plt.title('ODE Solution Approximation')
    plt.show()
\end{lstlisting}
\end{codebox}

\section{Results}

\subsection{Performance Metrics}
We evaluate the performance of our neural network solution using the following metrics:

\begin{table}[H]
\centering
\begin{tabular}{|l|c|}
\hline
\textbf{Metric} & \textbf{Value} \\
\hline
Mean Squared Error (MSE) & \(5.5021 \times 10^{-5}\) \\
Training Time & 1.6875 seconds \\
Maximum Absolute Difference & 0.0120 \\
\hline
\end{tabular}
\caption{Performance Metrics of the Neural Network-Based ODE Solution}
\end{table}

\subsection{Graphical Representation}
Figure \ref{fig:solution_comparison} shows a comparison between the neural network solution, the exact analytical solution, and the implicit Euler approximation.

\begin{figure}[H]
\centering
\includegraphics[width=\textwidth]{Figure_1.png}
\caption{Comparison of Neural Network, Exact, and Implicit Euler Solutions}
\label{fig:solution_comparison}
\end{figure}

\section{Conclusion}
This study demonstrates the effectiveness of neural networks in solving ordinary differential equations. Our approach, using a trial solution method with a feedforward neural network, successfully approximated the solution to the given second-order ODE.

Key findings include:
\begin{itemize}
    \item \textbf{High accuracy}: The neural network solution achieved a very low mean squared error of \(5.502 \times 10^{-5}\), indicating a close approximation to the exact solution.
    \item \textbf{Computational efficiency}: The model was trained in just 1.6875 seconds, demonstrating the potential for quick solutions to ODEs.
    \item \textbf{Competitive performance}: The maximum absolute difference between the neural network solution and the analytical solution was only 0.0120, showcasing the method's precision.
\end{itemize}

While the results are promising, there are areas for future research, such as:
\begin{itemize}
    \item Extending the method to more complex ODEs and systems of ODEs
    \item Optimizing the neural network architecture for better performance
    \item Comparing this approach with traditional numerical methods across a wider range of problems
\end{itemize}

As machine learning techniques continue to evolve, we anticipate that neural network approaches to solving differential equations will become increasingly sophisticated and widely applicable across various scientific and engineering domains.

\begin{thebibliography}{9}

\bibitem{Raissi2019}
M. Raissi, P. Perdikaris, and G. E. Karniadakis, 
\textit{Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations}. 
\textit{Journal of Computational Physics}, 378, 686--707, 2019.

\bibitem{Goodfellow2016}
I. Goodfellow, Y. Bengio, and A. Courville, 
\textit{Deep Learning}. 
MIT Press, 2016.

\bibitem{He2016}
K. He, X. Zhang, S. Ren, and J. Sun, 
\textit{Deep Residual Learning for Image Recognition}. 
In \textit{Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, 2016, pp. 770--778.

\bibitem{ODEBook}
L. N. Trefethen and D. Bau III, 
\textit{Numerical Linear Algebra}. 
Society for Industrial and Applied Mathematics, 1997.

\end{thebibliography}

\end{document}


can you check my assignemnt and see if facts and everything is correct or not