Innovative membrane technologies optimally integrated into large separation process plants are essential for economical water treatment and disposal. However, the mass transport through membranes is commonly described by nonlinear di ﬀerential-algebraic mechanistic models at the nano-scale, while the process and its economics range up to large-scale. Thus, the optimal design of membranes in process plants requires decision making across multiple scales, which is not tractable using standard tools. In this work, we embed artiﬁcial neural networks (ANNs) as surrogate models in the deterministic global optimization to bridge the gap of scales. This methodology allows for deterministic global optimization of membrane processes with accurate transport models – avoiding the utilization of inaccurate approximations through heuristics or short-cut models. The ANNs are trained based on data generated by a one-dimensional extended Nernst-Planck ion transport model and extended to a more accurate two-dimensional distribution of the membrane module, that captures the ﬁltration-related decreasing retention of salt. We simultaneously design the membrane and plant layout yielding optimal membrane module synthesis properties along with the optimal plant design for multiple objectives, feed concentrations, ﬁltration stages, and salt mixtures. The developed process models and the optimization solver are available open-source, enabling computational resource-e ﬃcient multi-scale optimization in membrane science. Keywords: Nanoﬁltration, Data-driven surrogate model, E nPE nmodel, Artiﬁcial neural network, Deterministic global superstructure optimization ˚These authors contributed equally to this work ˚˚ Corresponding author: manuscripts.cvt@avt.rwth-aachen.de 1 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 2 1. Introduction Global population growth and increasing urbanization require novel economically viable concepts for drinking water treatment and water disposal [1]. This challenge includes smart operations, such as data-driven urban water management [2], but also new treatment concepts and technologies [3, 4]. Synthetic membranes are an essential technological basis for this separation processes [5, 6]. Innovative membrane technologies help to develop new ap- proaches for the reuse of sustainable raw material sources from urban and industrial waste water [7, 8] as well as producing smart process water tailored to its application [9]. These ambitious aspirations require a versatile model- ing environment and rigorous optimization methodologies for membrane systems made for case-speciﬁc customized processes. Nowadays, ion selectivity becomes increasingly essential for sustainable drinking water treatment and electro- chemical processes [10, 11]. A transition followed from the complete removal of all minerals from drinking water through reverse osmosis membranes towards more selective nanoﬁltration membranes. The selectivity of ionic com- ponents enables the recovery of sustainable raw material sources from urban and industrial waste water [12], such as phosphorus [13]. The development of ion separation membranes is driven by the advent of the layer-by-layer (LbL) nanoﬁltration technology [14–16]. The layer-by-layer (LbL) nanoﬁltration technology is characterized by oppositely charged polymer layers applied to a conventional highly porous ultraﬁltration membrane. The layer-wise manufac- turing technique enables precise tuning of selectivity and permeability [17–19] and a scale-up from the laboratory to industrial scale [20]. However, tailoring of the selectivity of charged and uncharged species [21–23] is the predomi- nant challenge. Moreover, exploiting all synergies in membrane development through the simultaneous design of LbL membranes, the process structure, and its operation is an open research question [24]. Detailed mechanistic models describe the mass transport and separation by membranes at the nano-scale. There exists a variety of di ﬀerent membrane models for ion separation [25–29]. These mechanistic models range from highly accurate 3D models to low ﬁdelity heuristics, as shown in Figure 1. Most notably, a trade-o ﬀexists between the ﬁdelity of the approach and its 3D computational resource-e ﬃciency [30]. Accurate mechanistic models describe ion transport mechanisms in multiple dimensions, leading to partial di ﬀerential equations (PDE) that require special solvers for the solution, i.e., computational ﬂuid dynamics (CFD) for 3D. Lumping dimensions reduces the dimensionality to 2D and 1D models that can be evaluated at lower computational e ﬀort at the cost of ﬁdelity. Finally, heuristics describe lumped systems (0D) and can be assessed at little computational e ﬀort, but their predictive capability is insu ﬃcient to describe the complex ion transport. Moreover, only experiments obtain high ﬁdelity data directly. This results in the dilemma that increased accuracy leads to enormous computational e ﬀort and cannot be used in the context of process optimization by using standard tools. 2 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 3 Figure 1. ANN-based surrogate models push the boundary of computational recource-e ﬃciency: Fidelity and computational recource-e ﬃciency trade-oﬀof di ﬀerent descriptive approaches. The descriptive approaches include experiments, simulation studies, and heuristics. Adapted from [30] Economically viable and sustainable process design requires decision making at large process scales (potentially also considering environmental e ﬀects [31]). At the same time, physical models of the complex ion transport are described at small-scales. This multi-scale problem has strong implications for optimal decision making: On the one hand, superstructure process optimization with 3D, 2D, and 1D models embedded lead to (mixed-integer) optimization problems with PDE constraints. These mixed-integer dynamic optimization (MIDO) problems [32–34] are very di ﬃcult-to-solve and their deterministic global optimization is currently limited to small-scale problems [35]. Thus, process optimization using this approach is currently not possible. On the other hand, heuristics or short-cut models can be embedded computational resource-e ﬃciently in complex optimization superstructures [36–42]. However, these simpliﬁed heuristics or short-cut models do not su ﬃciently describe the inﬂuence of membrane parameters, such as the synthesis protocols or process environment, on the membrane performance. Simultaneous development of the membrane synthesis, along with the process using heuristics or short-cut models, is not possible. Overcoming the trade-o ﬀbetween ﬁdelity and computational resource-e ﬃciency is necessary for deterministic global membrane process optimizations with the simultaneous development of the membrane synthesis. One way to 3 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 4 overcome this issue is to replace the expensive simulation by a surrogate model [43]. Here, the accurate model is evaluated o ﬄine, creating a training data set that is subsequently learned by a supervised machine learning algorithm. This results in a data-driven surrogate model that approximates the accurate model. Then, the data-driven surrogate model can be combined with further data-driven and mechanistic models yielding a hybrid mechanistic /data-driven model that can be optimized [44]. The use of machine learning models enables bridging scales, e.g., in material design [45–50] or process systems engineering [51, 52]. In addition, data-driven models have been used in various disciplines for process optimization (e.g., [53–58]). Further, we have found that hybrid mechanistic /data-driven mod- els with artiﬁcial neural networks (ANNs) embedded can be optimized e ﬃciently by a reduced-space deterministic global optimization method [59]. In the past, this new optimization approach has already been used successfully for the optimization of energy processes [60–64]. Many publications in the ﬁeld of membrane science address the implementation of neural networks in membrane development [65–67] or prediction of membrane operation [68–70]. Recently, we extended the use of ANN surrogate models in membrane science to describe and optimize LbL-based membrane systems by identifying superior mem- brane synthesis protocols based on the delicate trade-o ﬀbetween ion retention characteristics and permeability [21]. Next, a coupling of the ANNs into a hybrid mechanistic /data-driven model enables an optimization strategy to simul- taneously design the performance of LbL nanoﬁltration membrane modules and the separation process. This yields membrane synthesis protocols and membrane processes that are optimally tailored to the desired separation task [24]. The results suggest that simultaneous membrane synthesis and process optimization design achieve immediately fa- vorable results with lower impurities at comparable costs. But in the previous work, this optimization has been limited to lab data. In particular, only a single-stage membrane process has been considered because the available experimen- tal data is only valid for a ﬁxed feed concentration and single salt solutions. Thus, an extension to accommodate for higher feed concentrations, multiple-staged processes, and salt mixtures is highly desired. In this work, we propose a surrogate-based approach to bridge the gap between mechanistic ion transport models at the nano-scale and optimal process design through deterministic global superstructure optimization at a large-scale. Thereby, we facilitate ANN surrogate models trained on data generated by a one-dimensional ion transport model, which are subsequently embedded in a state-of-the-art membrane process optimization model. We use the extended Nernst-Planck model, called pEnPEn, for describing the ion transport through the mem- brane [28, 71, 72]. The model pEnPEn describes ion transport through multi-layered geometries (i.e., LbL nanoﬁl- tration membranes) consisting of nelectrolyte layers (En) with npolyelectrolyte layers, i.e., membranes, (PEn). An applied pressure ( p) acts as the driving force of the separation process. The aforementioned one-dimensional ion transport model is evaluated o ﬄine, creating an extensive data set that describes the membrane’s ionic retention based on membrane-speciﬁc parameters (such as, the layer charge Xand the layer thickness ∆x) and process-speciﬁc pa- rameters (such as, the transmembrane velocity v, and ionic feed concentration cj). A subsequent training of the data creates an ANN surrogate model. 4 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 5 Next, the surrogate models are exploited towards a more accurate two-dimensional distribution of the membrane module even to capture the ﬁltration-related decreasing retention of salt. Therefore, individual ANNs are arranged in series to resolve the membrane in the direction of ﬂow. The data that is generated by this series of ANNs are used to create a two-dimensional surrogate model, which is then embedded in the membrane process optimization model. This enables the complex mechanistic model to be applied computational resource-e ﬃciently in an optimization context, despite the partial di ﬀerential-algebraic form of the original model. The paper is organized as follows: First, we present the data generation and the learning of ANN surrogate models. Second, the more accurate two-dimensional distribution of the membrane module is presented through ANN surrogate modeling. Third, the hybrid mechanistic /data-driven multi-scale process model is applied to a case study optimizing nanoﬁltration membrane plants for multiple objectives, feed concentrations, ﬁltration stages, and salt mixtures. The membrane module synthesis properties are optimized along with the superstructure of the membrane plant. In par- ticular, process conﬁgurations of the plant (i.e., number of ﬁltration stages, recirculating of streams, pump power) as well as the membrane synthesis properties (i.e., the layer charge Xand the layer thickness ∆x) are degrees of freedom and are optimized simultaneously. This work sets the foundation for computational resource-e ﬃcient multi-scale modeling integrating simulation studies of the complex ion transport in the decision making process at large process scale optimizations. Finally, all developed models and the optimization solvers are available open-source, making it a viable tool for future research and industrial applications. 5 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 6 2. Methodology In this section, the workﬂow of the data-driven approach is described to bridge the scales between (i) high ﬁdelity ion transport models on a nano-scale and (ii) deterministic global membrane process optimization on a large-scale. First, the high ﬁdelity simulation of the ion transport through the membrane on the nano-scale is replaced by a surrogate model. This surrogate model based replacement is performed by evaluating the high ﬁdelity ion transport model oﬄine and creating a training data set that is subsequently learned by a supervised machine learning algorithm (here ANNs). Then, the resulting data-driven ANN-based surrogate model that approximates the accurate model is embedded in a hybrid mechanistic /data-driven model and optimized to ﬁnd optimal membrane plant process conﬁgurations. In summary, this procedure avoids PDE constraints in the formulation of the deterministic global optimization problem. Thereby, this method enables deterministic optimization of membrane processes with surrogate models at a low computational cost similar to lumped systems (i.e., heuristics or short cut models) but signiﬁcantly more accurate. The workﬂow commences as depicted in Figure 2. (1) The high ﬁdelity ion transport model (Section 2.1) is eval- uated oﬄine to create an extensive data set, that (2) is used to train the data-driven ANN-based surrogate models on the data (Section 2.2). (3) Moreover, a surrogate model, including a two-dimensional distribution of the membrane module is created for higher ﬁdelity. The most precise two-dimensional distribution and additional data generation are described in Section 2.3, and the implementation of the two-dimensional surrogate model is described in Section 2.4. (4) Next, the individual surrogate models are integrated with a mechanistic process model to a hybrid mechanistic /data-driven process model. In Section 2.5, we explain the overall mechanistic process model, including cost cor- relations. (5) Finally, the hybrid model is optimized using a reduced-space formulation [59] and our open-source deterministic global solver MAiNGO [73]. In Section 2.6, we brieﬂy describe the optimization method. Figure 2. The workﬂow of the data-driven approach. The input data is created through the high ﬁdelity ion transport model simulation study. Next, an ANN takes this data to learn the ionic separation performance of the membrane based on the model, creating a data-driven ANN-based surrogate model. Two-dimensional distribution of the membrane module is constructed for higher ﬁdelity of the surrogate model. For this purpose, the surrogate model is used to generate additional data that is trained in a new ANN surrogate model. Next, this ANN surrogate model is embedded in a mechanistic process model. Finally, the optimal process design is solved by deterministic global process optimization. All tools necessary to rebuild and extend this method are open-source available to everyone. The process models and optimization problems are available open-source ( http://permalink.avt.rwth-aachen.de/?id=506639 ). The training data, the trained artiﬁcial neural networks, and the optimization results are available in the elec- 6 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 7 tronic supplementary material of this article (published by the journal). The artiﬁcial neural network models and training scripts are available open-source within the “MeLOn - Machin e L earning Models for Optimizatio n” tool- box [74] ( https://git.rwth-aachen.de/avt.svt/public/MeLOn/ ). Furthermore, the deterministic global solver MAiNGO is also available open-source ( https://git.rwth-aachen.de/avt.svt/public/maingo ). 2.1. Data generation by high ﬁdelity ion transport model In this work, the initial data set to train the ANN surrogate model is created by a simulation study using a high ﬁdelity mechanistic ion transport model. We use a one-dimensional extended Nernst-Planck model, called pEn- PEn [28, 71], to describe the ion transport through multi-layered geometries. LbL nanoﬁltration membranes can be modeled with this model [72] by introducing the layer thickness ∆xof the separation layer with a homogeneous layer charge Xas shown in Figure 3. Figure 3. Overview of model parameters of the extended ion transport model pEnPEn [28, 71, 72]. The layer charge X, and the layer thickness ∆xof the separation layer determine the separation performance. These values are inputs to the ANN and serve as a degree of freedom during simultaneous optimization of membrane module synthesis properties along with the superstructure of the membrane plant. The model is evaluated o ﬄine, creating a large data set that describes the membrane’s ionic retention based on membrane-speciﬁc parameters (such as, the layer charge Xand the layer thickness ∆x) and process-speciﬁc parameters (such as, the transmembrane velocity v, ionic feed concentration cj). The generation of data is automated, utilizing a wrapper script. This generates a set of training data inputs /outputs by a Latin hypercube sampling design for the ANN surrogate model. The variables are listed as follows: •The layer charge X[mol m´3]is a membrane-speciﬁc variable and input to the ANN. For single salts the layer charge varies in the interval r´ 500,´100s Ă R. For salt mixtures this interval needs to be extended to r´ 1000,´10 s Ă Rto account for the more challenging separation task. The membrane charge needs to be higher as compared to single salts due to the consideration of higher ionic concentrations for salts mixtures. 7 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 8 •The separation layer thickness ∆x[nm ]is a membrane-speciﬁc variable and input to the ANN. The separation layer thickness varies in the interval P r 75 ,150s Ă Raccording to experimental observations of [20]. •Thetransmembrane velocity v[µm s ´1]is a process-speciﬁc variable and input to the ANN. The retention of ions strongly depends on the transmembrane velocity, i.e., recovery rate, and varies in this study in the interval r0,50 s Ă R. •Thesalt feed concentration cj[mol m´3]is a process-speciﬁc variable and input to the ANN. The retention of ions strongly depends on the feed concentration. The range of salt feed concentration for a single membrane unit is r1,50 s Ă R. When using two membrane ﬁltration units in series, an extension of the data for feed concentrations below 1 mol m ´3is needed. Therefore, the data is extended using an additional Latin hypercube sampling to extend the data towards concentrations in the interval r0.01 ,50 s Ă R. •Theionic retention of ion j Rj[%] is the output to the ANN. The one-dimensional model is established for the membrane itself, along with two adjacent di ﬀusive layers. The model is implemented in COMSOL Multiphysics ®[75] and executed via the MATLAB [76] API. The partial di ﬀer- ential extended Nernst-Planck equation is simpliﬁed to a second-order ordinary di ﬀerential equation by considering only the steady-state behavior of the charged species. Moreover, the pressure-driven convective ﬂux is described via the ﬁrst-order ordinary di ﬀerential equation Darcys law. The boundary conditions are applied for the pressure at the inlet and outlet of the domain. Additionally, charge neutrality across the membrane is assumed. The resulting prob- lem formulation involves the coupling of two di ﬀerent ordinary di ﬀerential equations and is highly non-linear. It is solved with an adaptive Newton-Rapson algorithm. The systems matrix of the linearized problem is inverted using the MUMPS algorithm. This modeling approach is generic and thus applies to all combinations of salts used in this study. Due to convergence issues in the range of low initial salt feed concentrations (i.e., at high local gradients in the second-order ordinary di ﬀerential equation), a load ramping technique is applied to the excess charge parameter on the membrane surface. Input combinations to the one-dimensional ion transport model for which the solver did not converge are excluded from the training set. 2.2. Training of the data-driven surrogate model In this study, we utilize supervised machine learning techniques for the creation of surrogate models. In literature, there exists a large variety of machine learning techniques, including Gaussian processes, artiﬁcial neural networks, and decision trees for correlations of varying complexity. In this work, we implement ANNs as surrogate models, as they can capture the ion transport in membranes based on synthesis parameters, as demonstrated in our previous work [21, 24]. ANNs are black box models that are frequently used for unsupervised and supervised learning tasks, including pattern recognition, classiﬁcation, and regression [77]. ANNs learn training data and can approximate an 8 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 9 underlying input-output function. The evaluation of ANNs is given by an explicit algebraic function that depends on the architecture of the ANN, i.e., the connection of neurons [78]. This function is embedded in the process optimization problem and is e ﬃciently solved by a reduced-space deterministic global optimization method [59]. In this work, a shallow feed-forward multi-layered perceptron ANN including one hidden layer is utilized to recreate the one-dimensional extended Nernst-Planck mechanistic ion transport model (pEnPEn) from the previous section. A hyperbolic tangent transfer function is employed in the hidden- and output layer. The structure of the ANN is depicted in Figure 4. Inputs to the ANN are the layer charge X, the layer thickness ∆x, the transmembrane velocity v, and the salt feed concentration cj. Output to the ANN is the ionic retention of ion j Rj. Figure 4. Multi-layered perceptron ANN with one hidden layer used this study as a data-driven surrogate model to recreate the ion transport through nanoﬁltration membranes. The training is based on data generated by a one-dimensional extended Nernst-Planck mechanistic ion transport model (pEnPEn). The membrane-speciﬁc parameters (such as the layer charge Xand the layer thickness ∆x) and process-speciﬁc parameters (such as the transmembrane velocity vand salt feed concentration cj) are inputs to the ANN. The ionic retention of ion j Rjis the output to the ANN. The main drawback of the data-driven modeling approach using ANNs is its limited extrapolation capability. Meaning that any data-driven model is not valid anymore outside of its training data domain (i.e., its convex envelope). Therefore, this results in a large data requirement for the training of ANNs. In our study, the training data of the ANNs is generated by a mechanistic ion transport model that allows us to have any number of data points (within computational reason) for training and validation. A consideration of probabilities, i.e., through Gaussian processes, is omitted because we use a data set extracted from a mechanistic model. The training data set we use for this study is small compared to many other machine learning applications, such as deep artiﬁcial neural networks with millions of parameters that are frequently used for image recognition. Thus, we select a shallow artiﬁcial neural network architecture with one hidden layer ranging from 6 to 14 hidden neurons because the input dimensionality and training data set are much smaller. Furthermore, the training of the ANNs is optimized to minimize adverse e ﬀects using small 9 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 10 data sets including: (a) multiple runs of ANN trainings to mitigate sporadic ﬂuctuations in artiﬁcial neural network performance, (b) a randomized data set for training, validation and test to account for random e ﬀects due to small test data, and (c) the use of a k-fold cross-validation to prevent overshooting between data sets and avoid overﬁtting of the data set. The data preparation and training procedure are based on our previous work [21, 24], and adapted accordingly. The architecture of ANNs is ﬂexible and can be adjusted by setting its hyperparameters, e.g., number of layers, for application. In general, the architecture should be selected appropriately to avoid undesired under- and overﬁtting eﬀects. The available data set is split into 70 % training, 15 % validation, and 15 % test set at random. Early stopping is used to prevent overshooting during all ANN training. All ANNs used in this work are created in the MATLAB environment [76, 79]. The training procedure commences in the following steps: First, the hidden layer size is determined by k-fold crossvalidation with a range of nP r 1,50 s Ă Nhidden neurons. The minimum of the characteristic bias /variance trade-o ﬀbathtub curve of the mean-squared-error (MSE) determines the best number of hidden neurons nand thereby the hidden layer size. Second, ANNs are trained manifold with the previously determined hidden layer size n˘3 using a split of 70 % training, 15 % validation, and 15 % test set at random. Third, the best performing candidate of the ANN is selected for the following case studies. Two ANNs are utilized as one dimensional data-driven surrogate model considering a single salt Na 2SO 4or either a salt mixture of NaCl and Na 2SO 4. Here, only the predominant ion species (i.e., the ions Cl –and SO 2 – 4) of the ﬁltrated salt mixture are trained as inputs to the ANN. This consideration is valid due to electroneutrality in the solution and membrane. The monovalent cation Na +is calculated based on a mass balance. For single salts the ANN has one output (i.e., Na 2SO 4retention) and for salt mixtures, the ANN has two outputs (i.e., Cl –retention and SO 2 – 4 retention). Negative salt retentions are possible [80] and are mapped by the ANN. A list of properties of both ANNs is summarized in Table 1. Table 1. One-dimensional data-driven surrogate models used in this work to recreate the one-dimensional extended Nernst-Planck high ﬁdelity mechanistic ion transport model (pEnPEn). The number of neurons nof each ANN, the amount of training data set entries, and the section in which it is used is given. Network name Hidden neurons nDescription Training points Used in Section ANN Na 2S O 4,k“1,1 Unit 6 Single salt 998 2.3, 3.3 ANN NaCl ,Na 2S O 4,k“1,1 Unit 8 Salt mixture 1,000 2.3 2.3. Two-dimensional distribution of the membrane module Up until now, the membrane unit is considered where only the changes in the orthogonal direction of the mem- brane are considered by the one-dimensional surrogate model of the previous section. However, due to the retention of salt at the membrane, the feed concentration in the module increases with the length of the membrane. The feed con- centration of the salt strongly inﬂuences the retention of the nanoﬁltration membrane. Changes in feed concentration 10 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 11 in the direction of ﬂow are not considered so far. In Figure 5, a two-dimensional distribution of the membrane module is proposed in ﬂow direction through ANN- based surrogate models. Here, the membrane length is divided into several elements kand solved for the conservation equations on every element. An individual ANN describes each element. In Figure 5 A-B the strong inﬂuence of the feed concentration of the salt on the retention is evident for a selected membrane case. Figure 5. Two-dimensional distribution of the membrane module in ﬂow direction through ANN-based surrogate models. Shown is the negative eﬀect of the feed concentration on the salt retention of the membrane. Only the two-dimensional distributed cases do not underestimate this local reduction of ionic retention in the direction of ﬂow. In A), the retention of one selected case and membrane are shown for three di ﬀerent initial feed concentrations and varying element numbers k. In b), the corresponding feed sided concentration is shown resulting from the ﬁltration-related increase of feed concentration. Two e ﬀects contribute to the reduction of salt retention. First, the retention decreases with increasing initial salt feed concentration. This is demonstrated in Figure 5 A-B for the concentrations of 2 mol m ´3, 5 mol m ´3, and 10 mol m ´3. Second, a ﬁltration-related reduction of the retention of salt is observed in the direction of ﬂow. During 11 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 12 ﬁltration, the salt concentration increases at the retentate side. This negative e ﬀect of the increasing feed concentration on the salt retention of the membrane is di ﬀerently pronounced for the shown three cases. A one-dimensional dis- tributed membrane underestimates this e ﬀect and leads to inaccurate results. With an increasing number of elements k, the approximation approaches the most accurate concentration proﬁle. The relative gap between a one-dimensional distributed membrane and a two-dimensional distributed membrane is calculated for the speciﬁc case in Table 2. Here, the relative gap ranges from +43% to +240% dependent on the initial feed concentration. These results are case-speciﬁc but apply to any other case. Table 2. The Feed concentration increases along the membrane length during ﬁltration. The relative gap of the permeate concentration is calculated based on a two-dimensional distributed membrane with k“1000 elements and a one-dimensional distributed membrane with k“1 element for di ﬀerent feed concentrations. The gab occurs due to the decreasing retention with increasing feed concentration which is not accounted for in a one-dimensional distributed membrane. Variable name Unit cf eed =10 mol m ´3cf eed =5 mol m ´3cf eed =2 mol m ´3 Two-dimensional distributed ( k“1000) [mol m´3] 6.71 1.99 0.47 One-dimensional distributed ( k“1) [mol m ´3] 9.62 4.58 1.61 Absolute di ﬀerence [mol m´3] 2.92 2.53 1.14 Relative di ﬀerence [%] +43.50 +129.75 +239.86 Next, the most accurate number of elements for a single salt or salt mixture process optimization is identiﬁed. This is necessary to determine the number of two-dimensional distribution elements kthat are necessary to describe the change of states along the membrane surface accurately. Therefore, the number of elements is increased until the relative gap of the concentration is below 1%, where it can be considered su ﬃciently accurate. The relative di ﬀerence (or residuum) is calculated by the relative gap between the two-dimensional distributed ( ką1), and one- dimensional distributed ( k“1) permeate concentration leaving the membrane module. This procedure is repeated for di ﬀerent membrane conﬁgurations and observed covering the whole application range. Two exemplary cases and the progression of the residuum over the number of elements used are visualized in Figure 6. Here, a maximum number of k“56 elements are needed for single salt applications, and k“51 elements are required for salt mixtures. 12 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 13 Figure 6. Two-dimensional distribution of the module in ﬂow direction: The membrane unit is split into multiple elements k. The number of elements is increased until the relative gap of ﬁltration performance through the permeate concentration is below 1%. 2.4. Training of the surrogate model with two-dimensional distribution of the membrane Resolving the membranes through two-dimensional distribution is achieved by subdividing it into a ﬁxed number of elements and arranging the individual ANNs in series. Then, the mass conservation laws and the ANN computations are employed on every element. This creates a local output state from the input for each element. The technique is recursively applied to all elements along the direction of ﬂow. Now, this repeated computation of the local element quantities can be directly embedded into the process of plant optimization. However, this nesting of ANNs can lead to weak McCormick relaxations and can adversely aﬀect the computational performance of global optimization [81]. An alternative approach is to train a new ANN surrogate model that includes these recursions as a black box creating a single surrogate model for the two-dimensional modeling of the nanoﬁltration module. The training of an ANN-based surrogate model, including a two-dimensional distribution of the membrane module, is described in the following. A special case of data generation and ANN training arises when creating the surrogate model, including a two- dimensional distribution. The one-dimensional surrogate models trained on the one-dimensional extended Nernst- Planck ion transport model are connected serially one after the other and evaluated o ﬄine. This series of surrogate models create an additional data set for the new two-dimensional surrogate model. This extended two-dimensional surrogate model contains additional inﬂuential variables as the volume ﬂows of the entire module need to be con- sidered. A partial resolution of the recursive equation (exemplary for the single salt case) results in the following description 13 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 14 cin pkq “ cin pk´1q ¨ ˆ 1`γ K´ p k´1q ¨ γRpk´1q˙ (1) where cin ,jdenotes the feed concentration in element k,Kthe total number of elements, Rjthe ionic retention, andγthe ﬂow ratio which denotes the ratio of feed and permeate ﬂow along with the nanoﬁltration unit. Where γis deﬁned as γ“Fpermeate Ffeed. (2) This ﬂow ratio of the membrane module determines the performance of the unit. Therefore, the two-dimensional surrogate model is extended by the additional input of γto account for the extra parameter. For obtaining the training data of the two-dimensional surrogate model, an additional Latin hypercube sampling for a random two-dimensional distribution is applied, and the data created o ﬄine. Due to the low computational e ﬀort to calculate such recursive ANNs in series, k“1000 elements is chosen for the following process optimizations. Therefore, the relative di ﬀerence between the two-dimensional distributed ( ką1) and one-dimensional distributed (k“1) permeate concentration leaving the membrane module is always below 1%. For example, in Figure 6 only a maximum number of k“56 elements is needed. The training procedure is performed as previously stated in Section 2.2. Now, inputs to the ANN are the layer charge X, the layer thickness ∆x, the transmembrane velocity v, the salt feed concentration cj, and the additional parameterγfor the ﬂow ratio. The ﬂow ratio γis valid in the range P r 0.18; 1 s Ă Runder the consideration of the intercorrelation of γ(i.e., the permeate ﬂow can never be higher than the feed ﬂow). Output to the ANN is the ionic retention of ion j Rj. A list of properties of the new two-dimensional surrogate models can be found in Table 3. Table 3. Two-dimensional surrogate models used in this work to recreate the two-dimensional distribution of the membrane in the ﬂow direction. The number of neurons nof each ANN, the amount of training data set entries, and the section in which it is used is given. Network name Hidden neurons nDescription Training points Used in Section ANN Na 2S O 4,k“1000,1 Unit 6 Single salt 998 3.3, 3.4 ANN Na 2S O 4,k“1000,2 Units 6 Single salt 1,198 3.4 ANN NaCl ,Na 2S O 4,k“1000,1 Unit 14 Salt mixture 1,000 3.5 2.5. Hybrid mechanistic /data-driven process superstructure model The individual surrogate models are integrated with a mechanistic process model to a hybrid mechanistic /data- driven process model. The hybrid process superstructure model comprises of a mechanistic process model to describe the process plant and of surrogate models to describe the ionic transport based on membrane and process-speciﬁc 14 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 15 variables. The mechanistic process model consists of component mass balances, a pump model, and cost correlations. The superstructure of the process plant is shown in Figure 7. The model is adapted from our previous work [24]. Either a single-stage nanoﬁltration unit or a two-stage connection of nanoﬁltration units is chosen for the process plant. A pump is installed in the inlet stream of each nanoﬁltration unit to provide the main driving force. Each nanoﬁltration unit consists of multiple membrane modules, Nmodule in parallel. Each nanoﬁltration unit (unit i) has a feed stream Ff eed ,i. A concentrated retentate stream Fretentate ,iand a diluted permeate stream Fpermeate,ieach leave the nanoﬁltration unit. The permeate stream Fpermeate,iis calculated based on the membrane’s ionic retention performance, permeability, and pressure di ﬀerence∆papplied: Fpermeate,i“Q¨Aunit,i¨∆p (3) where Qis the permeability of the membrane unit, Aunit,i“Nmodule ,i¨Amodule is the total membrane area of the membrane unit. The size of a single membrane module in a nanoﬁltration unit is set to Amodule =60 m 2in accordance with multiple membrane modules available on the market [82, 83]. The permeability of the membrane is estimated based on a linear correlation of the membrane thickness ∆xwith the range QP r 5; 35 s Ă Rin accordance with laboratory results obtained in our previous work [20]. Q“0.4¨∆x´25 (4) Additionally, the layer charge Xis assumed to increase linearly with membrane thickness ∆x. A mass balance coupled with the ionic retention of ion j Ri,jfor ion jcloses the governing equations. Ff eed ,icf eed ,i,j“Fretentate ,icretentate ,i,j`Fpermeate,icpermeate,i,j (5) Ri,j“1´cpermeate,i,j cf eed ,i,j(6) The ionic retention of ion j Ri,jis computed by the ANNs as a function of membrane-speciﬁc variables and process-speciﬁc variables as described in Section 2.2 and Section 2.4. 15 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 16 Figure 7. Layouts of the membrane process plant consisting of either a single nanoﬁltration unit or a two-stage connection of nanoﬁltration units. Permeate ﬂow recirculation is allowed but does not have to be used. Each of the membrane units is equipped with a feed sided pump. A global feed, and retentate and permeate are connected by mixing units and represent the process input and outputs The cost model for this study is adapted from our previous work [24]. Here, the Verberne cost model [84–86] is used. The total investment cost Cinvestment is calculated based on the sum of the civil Ccivil, mechanical Cmechanical , electromechanical Celectro , and membrane Cmembrane investment cost Cinvestment “Ccivil`Cmechanical `Celectro `Cmembrane. (7) The annual operation cost is calculated based on the sum of the depreciation cost Cdepreciation , the energy cost Cenergy , the maintenance cost Cmaintenance , and speciﬁc cost Cspeci f ic . Coperation “Cdepreciation `Cenergy `Cmaintenance `Cspeci f ic (8) Appropriate cost parameters and depreciation periods are accounted for by the individual investment cost [24]. This study does not consider the e ﬀects of osmotic pressure di ﬀerences on the reduction of the driving force (here transmembrane pressure). This reduction of the driving force means that the presented results underestimate the pumping power because of the high salt concentrations used in this study. Considering the previously described cost model, it might be more economically viable to implement more membrane modules per membrane unit than using a higher pressurized feed stream. There always exists a trade-o ﬀbetween the costs by increased pumping power and the capital costs of more membrane area. Thus, the integration of the osmotic pressure into the proposed framework would be relevant and important for future research. In particular, future work could consider osmotic pressure di ﬀerences in the ion transport models by accounting for the osmotic pressure in the two-dimensional model 16 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 17 used for data generation (c.f. Section 2.3-2.4). However, this is beyond the scope of the current study. Considering the eﬀects of osmotic pressure di ﬀerences on the reduction of the driving force demand for a more detailed investigation as well as validation of the model against existing methods. 2.6. Numerical optimization approach Finally, the hybrid model is optimized using a reduced-space formulation and our open-source deterministic global solver MAiNGO [73]. The optimization problems in this work are formulated as a multi-objective mixed-integer nonlinear program (MINLP): min x,y¨ ˝f1px,yq f2px,yq˛ ‚ s.t.gipx,yq “ 0,i“1,...,I hjpx,yq ď 0,j“1,...,J xPXĂRn,yPYĂZm(9) where gipx,yqare equality constraints, hjpx,yqare inequality constraints, f1px,yq,f2px,yqare the objective functions, xare continuous optimization variables, and yis the integer optimization variable (i.e., number of membrane mod- ules Nmodule ,i). All model equations for the mechanistic process model are described in Subsection 2.5. The ionic separation performance is mapped by ANNs as described in Subsection 2.1, which are formulated in a reduced-space formulation [59]. The objective functions are (i) minimal annual operation costs ( f1px,yq “ Coperation ) and (ii) mini- mal permeate concentration ( f2px,yq “ cpermeate ). The MINLP is solved for the two objectives by the ǫ-constraint method. Here, the multi-objective problem is reformulated to multiple single-objective problems. Thereby, one objective is minimized, and the other objective is enforced to be less or equal to a parameter ǫ. This procedure is repeated for di ﬀerentǫyielding Pareto-optimal points. All optimization problems in this work are implemented and solved by our open-source deterministic global solver MAiNGO [73]. 17 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 18 3. Case study & results In this section, the results of the process optimizations, including the ANN-based surrogate models, are discussed for a speciﬁc case scenario (cf. Section 3.1). In Section 3.2 the methodology to integrate multi-scale membrane plant optimization using LbL nanoﬁltration membranes is presented. Then, the performance of the overall process is evaluated when including a more accurate two-dimensional surrogate model by an optimal two-dimensional distri- bution of the membrane in Section 3.3. Here, the single ﬁltration unit is considered by (A) using a one-dimensional distribution, (B) (1) by using a three-element two-dimensional distribution which is directly implemented in the op- timization framework, and (B) (2) by using a new surrogate model that includes these two-dimensional distributions over k“1000 cells in a black box creating a computational resource-e ﬃcient two-dimensional modeling of the nanoﬁltration module. In Section 3.4, the ﬁltration performance of the process is evaluated when including a second stage. Finally, in Section 3.5, salt mixtures are considered and the process optimized. 3.1. Case scenario for membrane plant optimization The case scenario and cost correlations are adapted from our previous work [24]. The membrane plant is op- timized based on a drinking water puriﬁcation meeting given quality speciﬁcations for a small town with 10 ,000 inhabitants [87]. In this study, water softening is considered by retaining Na 2SO 4salt in the membrane plant. The drinking water demand amounts to a peak demand of approximately 224 m 3h´1. All processes dimensions are set to meet the permeate volume ﬂow of this peak demand. Demand-side management is not considered within the scope of this work. 3.2. Bridging the gap of multiple scales by surrogate model-based membrane plant optimization In this section, the results of the surrogate model-based membrane plant optimization are presented. The opti- mization is performed for a single-stage nanoﬁltration unit (cf., Figure 7) using the ANN Na 2S O 4,k“1,1 Unit surrogate model (cf., Table 1) to describe the ionic retention of the membrane unit. The whole process is optimized for ranging feed concentrations of Na 2SO 4and solved for the two objectives – (i) minimal annual operation costs and (ii) minimal Na 2SO 4permeate concentration. The Na 2SO 4feed concentration range from 5 mol m ´3to 20 mol m ´3. The results of this optimization are shown in Figure 8 depicting a smooth, convex Pareto front. 18 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 19 Figure 8. Surrogate model-based membrane plant optimization for a single-stage nanoﬁltration unit using the ANN Na 2S O 4,k“1,1 Unit surrogate model to describe the ionic retention of the membrane unit for Na 2SO 4feed concentration ranging from 5 mol m ´3to 20 mol m ´3. The optimiza- tion results in a smooth, convex Pareto front. The annual operating costs increase along with higher purity and with the more di ﬃcult ﬁltration task. Ideally, an optimal membrane plant would operate at zero costs and zero permeate concentrations for all feed concentrations. This is called the utopia point. However, in reality, higher feed concentrations (i.e., more di ﬃcult ﬁltration) and lower permeate concentrations (i.e., higher purities) always lead to higher operational costs. Hence, we face an inherent trade-o ﬀbetween conﬂicting objectives. This trade-o ﬀcan be addressed by multi-objective optimization. The solution of a multi-objective optimization problem is a Pareto-front between the objective functions. The points on the Pareto front correspond to processes where neither one of the objectives can be improved anymore without worsening another objective. Thus, for each of the Pareto e ﬃcient process points, one cannot archive higher purity at a given feed concentration without spending the additional operational cost. Considering the calculated Pareto front in Figure 8, indeed the annual operating costs increase along with higher purity (i.e., lower Na 2SO 4permeate concentration) and more di ﬃcult ﬁltration task (i.e., higher Na 2SO 4feed con- centration concentration). The achievable permeate purity strongly depends on the process’ salt feed concentration. Here, the maximum process purity ranges from 0.63 mol m´3for a feed concentration of 5 mol m ´3to 3.6 mol m ´3 19 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 20 for a feed concentration of 20 mol m ´3. The latter is the most di ﬃcult separation task of this optimization resulting in an annual operation cost of approximately 700 thousand eyear´1. A table of the optimal solution points can be found in the supplementary data to this publication. In the presented work, we demonstrate that nano-scale ion transport models can be used in process optimization. Besides, some essential e ﬀects for the design of membrane processes result from the observation of the simulated data. Therefore, the qualitative di ﬀerences between the process conﬁgurations are assessed in the following. The tables to this discussion can be found in the supplementary data to this publication. The optimal process conﬁguration towards higher purity (and higher annual operation cost) follows a stringent pattern regarding the involved process parameters. The process recirculation (cf., Figure 7 single stage) is fully exploited to minimize the ﬁltration units feed concentration. Permeate ﬂow recirculation is used to keep the feed concentration low to maintain the membrane’s re- tention high. This process indeed seems to be non-intuitive because the puriﬁed permeate stream is recirculated back to the feed stream. However, the feed concentration strongly inﬂuences the retention of the nanoﬁltration membrane and must, therefore, remain low to sustain high ionic retention of the membrane. It becomes apparent that it is eco- nomically more viable to recirculate the permeate stream than investing in more membrane area or higher membrane charges associated with low permeability. Moreover, the membrane thickness and surface charge are also increased throughout the optimization to yield a similar, but quantitatively less signiﬁcant, e ﬀect. The increase of the membrane thickness leads to a decreased permeability, which has to be compensated by a larger membrane surface or higher pressure gradient. This can be observed from a large number of membrane modules utilized for high-purity pro- cess conﬁgurations. Additionally, a large membrane surface facilitates a suitable mass transfer along with a medium transmembrane velocity, which also has a signiﬁcant impact on retention. For processes with lower cost and purity, the membrane’s surface is drastically reduced to eliminate this cost factor. In particular, no recirculating is performed to minimize the operation costs for the involved feed-sided pressure pump. The preceding observations apply to all following discussed process conﬁgurations. For the ﬁltration of salt mixtures, small restrictions have to be made that shall be considered later. Furthermore, the shape and position of the calculated Pareto front can be used to determine the performance of the membrane system. In Figure 8, a curved Pareto front towards the utopia point indicates overall a better performance. Whereas a curved Pareto away from the utopia point means a worse performance of the process up to the angular end of the feed- and permeate concentration plane so that no ﬁltration takes place. This visual e ﬀect may be used when comparing di ﬀerent membrane systems embedded in the same process of plant optimization. Taking a closer look at the next Section 3.3, the shape and position of the calculated Pareto front indicates the severe underestimation of the costs when only including a one-dimensional distribution of the membrane. The optimization problem includes 6 optimization variables, 1 equality constraint, and 9 inequality constraints. The optimization was executed on 4 cores in parallel on a high performance computing cluster. The average CPU time summed over all cores for the optimization of a Pareto point was 2.1 CPU seconds (see Table 4). Thus, the computation of the Pareto front, which is approximated by 100 points, took roughly 35 CPU minutes in total. 20 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 21 3.3. Optimization including the two-dimensional distribution of the membrane In the next step, the performance of the overall process is evaluated when including a more accurate two- dimensional surrogate model by an optimal two-dimensional distribution of the membrane. A membrane module is considered by (A) using a one-dimensional distribution as presented in the previous Section 3.2), and compared to (B) (1) using a three-element two-dimensional distribution by three ANN Na 2S O 4,k“1,1 Unit surrogate models (cf., Table 1) which are directly implemented in the optimization framework, and (B) (2) by using a new surrogate model ANN Na 2S O 4,k“1000,1 Unit (cf., Table 3) that includes these two-dimensional distributions as a black box creating a computational resource-e ﬃcient two-dimensional modeling of the nanoﬁltration module. Again, one single-stage membrane unit (c.f., Figure 7) is considered for the optimization of a process plant for ranging feed concentrations of Na 2SO 4and solved for the two objectives - (i) minimal annual operation costs and (ii) minimal Na 2SO 4permeate concentration. The resulting Pareto fronts of the optimizations are shown in Figure 9. Here, for comparability, the results of Figure 8 are displayed in Figure 9 (Membrane not distributed, i.e. k“1) as well. A table of the optimal solution points can be found in the supplementary data to this publication. Figure 9. Surrogate model-based membrane plant optimization for a single-stage nanoﬁltration for Na 2SO 4ﬁltration. Two implementations of the two-dimensional distribution are presented: (B) (1) by direct implementation of the two-dimensional distribution by three ANN Na 2S O 4,k“1,1 Unit surrogate models (B) (2) by the recursive back box surrogate model approach using ANN Na 2S O 4,k“1000,1 Unit . For comparability, the results of Figure 8 are displayed using one ANN Na 2S O 4,k“1,1 Unit surrogate model with one-dimensional distribution. Showing A) the results for a feed concentration of 10 mol m ´3and B) for ranging feed concentrations between 5 mol m ´3and 20 mol m ´3. (B) (1) Directly embedding the two-dimensional distribution in the optimization framework results in a dramatic increase in run time for only a few two-dimensional distribution elements. As shown in Table 4, the average CPU time to solve one Pareto point using k“3 elements is 9 .1¨10 4CPU seconds corresponding to over 25 CPU hours. To reduce the wall-clock time, we solved this optimization parallel on 48 cores. (Note that the CPU times are summed over all cores.) Thus, the maximum number of elements for directly embedding the two-dimensional distribution in 21 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 22 the optimization framework is k“3 elements using a single nanoﬁltration unit. This low number of elements is not su ﬃcient to accurately describe the changes along the membrane length, as presented in Section 2.3. The results suggest clearly that the solution of a large number of elements, i.e., ką3, would not be feasible using this method. (B) (2) In contrast, our proposed recursive black-box approach using the ANN Na 2S O 4,k“1000,1 Unit surrogate model requires on average 2 .4¨10 2CPU seconds to solve a Pareto point (summed over all cores). Thus, the complete Pareto front, including 100 points, is solved within 7 CPU hours. As we use 48 cores in parallel, this corresponds to a wall- clock time of under 10 minutes. Thus, the method signiﬁcantly reduced the computational e ﬀort compared to (B) (1) and makes the integration of k“1000 elements feasible. Compared to the consideration of k“1, we can observe an acceptable increase in CPU time, which is mainly due to the additional input of the ANN describing the ﬂow ratio γ. Table 4. Computational performance comparison of the proposed optimization methods. For each method, 100 optimization problems are solved, i.e., one for each Pareto point. Not all solved points are displayed in the ﬁgure. We provide average, variance, minimal, and maximal CPU times overall optimization problems. Note that CPU times for optimization include preprocessing and branch-and-bound time summed over all parallel cores in seconds. CPU time summed over all cores [s] # Name Section k [-] CPU Cores Average Variance min max A Na 2S O 4,k“1,1 Unit 3.2 1 4 2.1¨10 12.4¨10 27.7¨10 ´15.6¨10 1 B (1) Na 2S O 4,k“3,1 Unit 3.3 3 48 9.1¨10 42.5¨10 10 2.3¨10 07.7¨10 5 B (2) Na 2S O 4,k“1000,1 Unit 3.3 1000 48 2.4¨10 21.1¨10 51.7¨10 01.5¨10 3 A severe underestimation of the costs is observed in Figure 9 A when not including a two-dimensional distribution of the membrane. This severe underestimation of the costs is prevailing for all feed concentration of Na 2SO 4from 5 mol m ´3to 20 mol m ´3as indicated by the location of the three Pareto surfaces shown in Figure 9 B. The use of a more precise two-dimensional distribution of the membrane reveals a worse performance than the idealized process only with one-dimensional distribution in Figure 9. The worsening in process performance can be explained by the ﬁltration-related decreasing retention of salt at the retentate side. For positive retention (which is always the case for the single salt ﬁltration examined here), the concentration on the retentate side of the ﬁltration unit steadily increases along the membrane surface. However, this leads to decreasing retention and, therefore, a worse ﬁltration performance. Additionally, a back-coupling through recirculation of the permeate feed stream back to the membrane’s unit feed stream is observed. For the two-dimensional distribution of the membrane with k“1000 elements, for instance, the maximum achievable performance for a feed concentration of 5 mol m ´3is 1.19 mol m ´3and therefore presents an increase in impurity as compared to the idealized case using the one-dimensional distribution. This e ﬀect is even more pronounced when considering higher feed concentrations. 3.4. Extension of the optimization process by a second ﬁltration unit Next, the two-dimensional surrogate model approach is utilized to extend the optimization process by a second ﬁltration unit (cf,. Figure 7 two stage). The ANN Na 2S O 4,k“1000,1 Unit surrogate model is taken from the previous 22 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 23 Section 3.3. For the two-stage process the ANN Na 2S O 4,k“1000,2 Units surrogate model is chosen (cf., Table 3). The whole process is optimized for ranging feed concentrations of Na 2SO 4from 5 mol m ´3to 20 mol m ´3and solved again for the two objectives - (i) minimal annual operation costs and (ii) minimal Na 2SO 4permeate concentration. The results of the optimization are shown in Figure 10. A table of the optimal solution points can be found in the supplementary data to this publication. Figure 10. Surrogate model-based membrane plant optimization for a single-stage and two-stage serial connection of nanoﬁltration units for Na 2SO 4ﬁltration. The two-dimensional surrogate model approach approach is chosen using ANN Na 2S O 4,k“1000,1 Unit for one nanoﬁltration unit, and the ANN Na 2S O 4,k“1000,2 Units surrogate model for the two-stage serial connection. Showing A) the results for a feed concentration of 10 mol m ´3and B) for ranging feed concentrations between 5 mol m ´3and 20 mol m ´3. This combination of a two-stage series of ﬁltration units results in a strongly enhanced ﬁltration performance. This tendency is particularly intensiﬁed as the feed concentration of the second unit is already the puriﬁed concentration of the ﬁrst unit. Due to the strong correlation of the feed concentration and retention, the second ﬁltration unit is particularly useful. Notably, the improvement is much more signiﬁcant for lower feed concentrations. For higher initial feed concentrations, however, the e ﬀect of utilizing two ﬁltration units is not as predominant. There exists a distinct switching point, as highlighted in Figure 10 A, where the economic viability of either one or two nanoﬁltration unit changes. In the range of less di ﬃcult ﬁltration operations, a single-stage option is preferable. More di ﬃcult ﬁltration operations with higher purity, however, are cheaper under the inclusion of a second unit (or even impossible for a single unit). The costs of a second ﬁltration unit fall short in the overall economic estimation. This distinct switching point is case-speciﬁc and dependent on the feed concentration of Na 2SO 4. As shown in Figure 10 B for all feed concentration of Na 2SO 4from 5 mol m ´3to 20 mol m ´3a distinct switching point exists highlighted by the change of color (here from black to purple). The superstructure optimization includes 12 continuous and 2 integer optimization variables, 6 equality con- straints, and 14 inequality constraints. We run the optimization for the two-unit superstructure with k“1 elements 23 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 24 per unit on 48 cores (result not displayed in Figure 10). Here, the average CPU time summed over all cores for one Pareto point was 7 .7¨10 3seconds. This is an increase by a factor of 360 compared to the one unit process with a k“1 element. Due to the increased computational e ﬀort, the results with two units and k“1000 have been solved using a local optimization approach with 10 multistarts. This method cannot guarantee global optimality. Therefore, only the results with two units and k“1 can guarantee global optimality, but are not meaningful as they severely underestimate the overall process costs. The local optimization took on average 25 CPU seconds for a Pareto point. 3.5. Extension of the optimization process by salt mixtures Next, the two-dimensional surrogate model approach is extended by salt mixtures. Due to computational e ﬀort, only one single-stage is considered in the optimization (cf,. Figure 7). The ANN NaCl ,Na 2S O 4,k“1000,1 Unit surrogate model is utilized in the process optimization framework. The whole process is optimized for ranging feed concentra- tions of Na 2SO 4from 5 mol m ´3to 20 mol m ´3. The feed concentration is extended by the addition of NaCl by ﬁxed composition ratios. More precise the ratios for NaCl:Na2SO 4are 1:2, 1:1, and 2:1. For example in Figure 11 A, for the ratio NaCl:Na2SO 4=1:2, 5 mol m ´3of NaCl is added to the feed stream when considering a feed of 10 mol m ´3 Na 2SO 4. The optimization is again solved for the two objectives - (i) minimal annual operation costs and (ii) minimal Na 2SO 4permeate concentration. The results of the optimization for the given ratios for NaCl:Na 2SO 4are shown in Figure 11. A table of the optimal solution points can be found in the supplementary data to this publication. To reduce computational e ﬀorts, the optimization problem is solved for fewer operating conditions (Pareto pints), leading to a more coarse approximation of the Pareto front. The shape and di ﬀerent process conﬁgurations show qualitatively similar behavior concerning the optimization of the single salt processes in Figure 9. In particular, the operational costs are in the same order of magnitude for both cases. Taking a closer look at the di ﬀerent Pareto fronts for the individual salt mixtures, they reveal di ﬀerences in ﬁltration operation. On the one hand, the Pareto fronts for ratios 1:1 and 2:1 (NaCl:Na 2SO 4) in Figure 11 C-D present similar behavior to the previous results. On the other hand, the optimization points for a ratio of 1:2 in Figure 11 B are clustered in a much narrower region of feasible points in the Pareto front. This process infeasibility is mainly observed for higher feed concentrations. This behavior can be ascribed to the more complex relations between the membrane retention, and its inﬂuencing process-speciﬁc parameters. These relations are neither monotone (as for the single salt case), nor is the retention itself limited to positive values (the Donnan e ﬀect induces negative retention). Most notably ﬁltration operation with high retention is still possible in Figure 11 D even at very high salt concentrations (i.e., 40 mol m ´3of Na 2SO 4and 20 mol m ´3of Na 2SO 4) by considering the Donnan e ﬀect. Hence an accurate description of the underlying physical processes becomes quite cumbersome, making the surrogate-based model for the determination of optimal operation points even more valuable. The increased complexity of the process for salt mixtures also manifests in clustered points on the Pareto front. Such points are Pareto-dominated by preceding conﬁgurations, which means a process in that particular setting gets more expensive under looser purity constraints. 24 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 25 Figure 11. Surrogate model-based membrane plant optimization for a single-stage nanoﬁltration unit for ﬁltration of NaCl and Na 2SO 4salt mixture. The two-dimensional surrogate model approach is chosen using the ANN NaCl ,Na 2S O 4,k“1000,1 Unit surrogate model. Showing A) the results for a Na 2SO 4feed concentration of 10 mol m ´3and for ranging feed concentrations between 5 mol m ´3and 20 mol m ´3for salt mixture ratios (NaCl:Na2SO 4) of B) 1:2, C) 1:1, and D) 2:1. The optimization for salt mixtures includes 6 continuous and 1 integer optimization variables, 2 equality con- straints, and 13 inequality constraints. We run the optimization for one single-stage unit and k“1000 elements on 48 cores. The average CPU time summed over all cores for one Pareto point was 3 .3¨10 3, 1 .1¨10 5, and 1 .0¨10 4 seconds for 1:1, 1:2, and 2:1 salt mixtures (NaCl:Na2SO 4), respectively. These results demonstrate that the CPU time for optimization is not only dependent on the structure of the optimization problem but also the case study. 25 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 26 4. Conclusion We present a computational resource-e ﬃcient methodology to integrate multi-scale modeling in membrane sci- ence. We propose a machine learning approach to integrate accurate physical models of ion transport, valid on the nano-scale, into large-scale superstructure optimization of membrane processes. Thereby, the multi-scale decision- making process allows us to simultaneously design the membrane synthesis properties along with the process design and operation. All models and the solver itself are open-source and freely available, enabling multi-scale modeling in membrane science. In particular, we train ANN surrogate models on simulated data from a di ﬀerential-algebraic one-dimensional extended Nernst-Plank ion transport model, called pEnPEn. These surrogate models are then exploited to extend the framework towards a more accurate two-dimensional distribution of the membrane module to capture the ﬁltration- related decreasing retention of salt. Here, individual ANNs are arranged in series to resolve the membrane in the direction of ﬂow. Next, the two-dimensional surrogate models are embedded in a mechanistic membrane process model leading to a hybrid mechanistic /data-driven model. The results demonstrate that: •A consideration of a one-dimensional distribution of the membrane module leads to a severe underestimation of process costs. Only the extension towards a two-dimensional distribution captures the adverse e ﬀects of ﬁltration-related decreasing retention of salt. •Non-intuitive interconnections of the membrane plant layout are optimal for high permeate purities, in which the permeate ﬂow recirculates back to keep the feed concentration low. •A distinct point between a single-stage and a two-stage process exists, where the economic viability of either one or two nanoﬁltration unit changes. Costs of a second ﬁltration unit fall short in the overall economic estimation for more di ﬃcult ﬁltration operations. •For salt mixtures, it is much more complex to ﬁnd optimal operating points, as further e ﬀects (i.e., Donnan exclusion) have an additional impact on the separation process. Therefore, it is all the more important that surrogate-based modeling bridges the gap between the scales for application of high ﬁdelity ion transport models in superstructure optimization of membrane processes. This work stimulates to exploit this open-source optimization framework by further research and in industrial applications, such as in drinking water treatment and water disposal. Additional surrogate models may include other membrane-based platforms. Furthermore, the concept of surrogate-based modeling should be exploited to integrate driving force diminishing e ﬀects acting on nano-scale, such as the osmotic pressure di ﬀerence. Future work in de- terministic global optimization and model reduction is desirable to push the computational limits of this approach further. 26 D. Rall et al. /Journal of Membrane Science 00 (2020) 1–31 27 Acknowledgement D.R., E.E., and M.W. acknowledge the support through the German Federal Ministry of Education and Research (BMBF) under the project UO-Rohrfabrikation (03XP0100E) and the project E ﬄueNF (02WIL1486) and the Eu- ropean Union’s Horizon 2020 research and innovation program (grant agreement no. 694946). A.M.S. and A.M. gratefully acknowledge funding by the excellence initiative of the German federal and state governments and they acknowledge the ﬁnancial support of the Kopernikus project SynErgie by the Federal Ministry of Education and Re- search (BMBF) and the project supervision by the project management organization Projekttr ¨ager J ¨ulich (PtJ). Funded by the Excellence Initiative of the German federal and state governments.