Synthetic membranes for desalination and ion separation processes are a prerequisite for the supply of safe and su cient drinking water as well as smart process water tailored to its application. This requires a versatile membrane fabrication methodology. Starting from an extensive set of new ion separation membranes synthesized with a layer-by-layer methodology, we demonstrate for the ﬁrst time that an artiﬁcial neural network (ANN) can predict ion retention and water ﬂux values based on membrane fabrication conditions. The predictive ANN is used in a local single-objective optimization approach to identify manufacturing conditions that improve permeability of existing membranes. A deterministic global multi-objective optimization is performed in order to identify the upper bound (Pareto front) of the delicate trade-o between ion retention characteristics and permeability. Ultimately, a coupling of the ANN into a hybrid model enables physical insight into the inﬂuence of fabrication conditions on apparent membrane properties. Keywords: Layer-by-layer, Nanoﬁltration, Artiﬁcial neural network, Upper bound, global optimization 1. Introduction Biological systems require lipid bi-layers and proteins to regulate transport of ions and natural molecules. Bio- logical membranes, their functional elements, their lateral dynamics, their mechanical properties, and their interfacial properties attracted signiﬁcant research of many communities. In the 1960s, biological membranes inspired the am- bition to develop artiﬁcial synthetic membranes to desalinate water for drinking water production. In the following decennia the community experienced a schism: while more and more molecular details could be identiﬁed for bi- ological membranes, the synthetic membrane community focused on high salt retention at high water permeation rates [1, 2, 3]. This research e ort has led to today’s reverse osmosis industry enabling safe and su cient water Corresponding author: manuscripts.cvt@avt.rwth-aachen.de 1 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 2 supply in arid regions. Also the use of speciﬁcally tailored ionic composition of process water - known as smart water - currently opens up new opportunities to access fossil resources that had to be considered not exploitable in the past [4]. As opposed to high-ion rejection in reverse osmosis, ion selectivity becomes increasingly important in the ﬁeld of sustainable water production [5, 6]. Only during the past two decennia the ﬁeld of nanoﬁltration has emerged with the main application to soften water, i.e. to rejection of divalent ions such as Ca2+and Mg2+. With the advent of bio- renewable-based chemical industry [7], as well as growing food demand, recovery concepts for valuable species, such as phosphorus [8], need to be developed. Energy storage and production systems frequently rely on the membrane’s property to distinguish between di erent ions [9]. This quest for ion separation properties requires a new paradigm in design of synthetic membranes with control of ion selectivity over a wide range of process parameters being the major scientiﬁc challenge. The development of nanoﬁltration membranes still relies on a large number of experiments, done by conventional screening procedures in an educated guess and experimental design context. Some innovative approaches have been proposed for the more rigorous development of membrane fabrication conditions. For example, a new facile toolbox was at hand with the advent of layer-by-layer (LbL) architectures comprising oppositely charged polyelectrolytes for the synthesis to explore the complex domain of ion transport under the inﬂuence of pressure gradients and electric ﬁelds [10, 11]. The LbL based nanocomposite nanoﬁltration membrane tool-box emerged as viable method to tailor membrane transport properties, the development of tailored nanoﬁltration membranes suited for a speciﬁc separation task remained costly [12]. Our recent development of coating inside an existing module [13], as well as regeneration of the module after a certain life time alleviates the cost constrained [14] and makes this synthetic method viable for scalable production. Hence, current research can now explore and unravel ﬁnally the relationship between physical properties and membrane fabrication conditions. Due to the complex transport phenomena at the membrane surface and within the membrane, the membrane transport properties strongly depend on the operating conditions. On top of the complex transport behavior, the properties of the LbL do not remain static but adapt to the ionic environment of the feed solution [15, 16]. This makes it still di cult to correlate the membrane’s transport properties to its chemistry and macro molecular architecture. Resolving the molecular-scale properties of the tailored LbL architectures is still a challenging task: the translation of such layered membrane structures into membrane transport models is in its infant state only [17]. Tailoring membrane fabrication conditions to speciﬁc separation tasks and understanding its membrane architectural consequences is today an unresolved fundamental task. In membrane science, trade-o relationships between productivity and selectivity have been established in the past. In gas separation, an upper bound challenges many polymer chemists [18]. In ultraﬁltration, the upper bound exists for membranes produced by the phase inversion process, and membrane engineers try to surpass this by using di erent fabrication methods [19, 20]. Here we demonstrate that such an upper bound exists for nanoﬁltration membranes made from polyelectrolytes based on a novel global deterministic optimization algorithm. In this work we present 2 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 3 two methodologies that identify the membrane production condition leading to increased ionic retention at increased pure water permeability. Ultimately, an extension of the methodology from a pure data-driven approach to a hybrid model enables physical insight into the build-up of the separation layer. The hybrid model consists of a combination of a state-of-the-art membrane transport model linked through model-based membrane properties to our previously described ANN methodology. The interface of the hybrid model is shown in Figure 1 (Hybrid Model). Figure 1. Overview of the scope of the presented methodology. Schematic representation of one of 10 membranes in one of 63 Modules with polyelectrolyte bi-layer applied on a porous hollow ﬁbre support [13]. (ANN Model) Synthesis of the membrane fabrication conditions based on desired membrane performance. (Hybrid Model) Hybrid form of the ANN model generating physical insight into the fabrication process. 3 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 4 2. Background 2.1. Nanoﬁltration membranes Nanoﬁltration represents a promising solution to the challenge of membrane ion separation [21, 22, 23, 24]. It was early on recognized that less dense reverse osmosis membranes are able to discriminate between monovalent and divalent ions. Nanoﬁltration membranes are mainly employed in water and waste-water treatment, removal of pollu- tants from ground water, solvent recovery, biotechnology, and pharmaceutical applications [25, 26, 27]. The synthetic principle to form nanoﬁltration membranes is based on the industrial production standard of interfacial polymeriza- tion through polycondensation reactions at an oil-water interface [28]. Monomer choice, reaction conditions, and additives determine ﬂux and retention properties. Exploring this design space is generally based on an evidence-based experimental e ort. 2.2. Layer-by-Layer (LbL) nanoﬁltration membranes LbL-based membranes were ﬁrst introduced after the advent of layered polyelectrolyte composites [29, 30, 31]. This manufacturing technique enables a precise tuning of the membrane performance regarding the desired selectivity towards one or more components. This groundbreaking work has inspired a whole community to explore the inﬂuence of polyelectrolyte character etc. on ion separation properties. In almost all cases, the LbL assemblies are prepared by adsorbing polyelectrolytes onto an emerging layer. Recently, we described a convective method that grants highly reproducible and tuneable properties [13]. With this method, polyelectrolyte bi-layers are adsorbed on the lumen surface of a porous hollow ﬁber support. A schematic of one of the fabricated LbL membrane used in this work is shown in Figure 1. The method was even versatile enough to be applied not only on a single membrane but also on membranes being packaged into a module housing. This makes the convective methodology scalable towards industry scales. Here we present an extensive set of 63 newly synthesized LbL-based membranes with di erent layer properties. The membranes were fabricated in modules having 10 ﬁbers. They were characterized by molecular weight cut-o (MWCO) measurements and salt retentions. The generated data of solvent permeability, ion retention and size exclusion in combination with the synthesis conditions serves as the data set for this work. Today, a methodology is missing that allows the prediction of the membrane transport properties based on LbL-fabrication conditions. This remains a challenging task, since the build-up of the layers depends on the polyelectrolyte type, its molecular weight, the ionic strength, pH, and concentration of the coating solution, the adsorption time, coating method, as well as the number of layers applied [32, 33, 34, 35, 36, 37]. 2.3. Modelling of nanoﬁltration membranes Quantifying the characteristics of membrane transport by means of mathematical modelling has been a subject of research since the 1960s in order to predict the membrane’s mass transport behavior. However, a model-based method- ology for the synthesis of tailor made membranes with predeﬁned retention for various ions and water permeability 4 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 5 has not been successfully introduced. Most advancements in the ﬁeld of nanoﬁtration membrane modelling have been achieved through transport models based on the Extended Nernst-Planck equation [38, 39, 40, 41, 42, 43]. These sev- eral contemporary models represent the membrane’s mass transport behavior using model-speciﬁc ﬁtting parameters. Yet, the relation of the ﬁtting parameter with the physical membrane properties as well as their correlation with the membrane fabrication conditions remains unexplored territory. No studies exist which explore a rich and complex synthetic parameter space and extracts physical properties from the salt and water transport properties. In this work, we explore the use of artiﬁcial neural networks (ANN) for two novel and unique purposes: (a) to predict transport, optimize membrane fabrication conditions and predict an upper bound for the rejection and permeation trade-o and (b) to extract and predict physical properties of membrane architectures depending on the synthetic conditions. 2.4. Neural network modelling in membrane science Recent advances in machine learning present a convincing alternative to phenomenological models. Inspired by the function of the human brain, ANNs are capable of correlating variables showing complex highly nonlinear depen- dencies [44]. ANNs have shown to be a versatile tool to classify data, recognize pattern, and approximate functions in multidisciplinary ﬁelds of research [45, 46, 47, 48, 49]. Thus far, ANN modelling in membrane science has been used to predict membrane operation, fouling behavior of the membranes and gas permeability [50, 51, 52, 53]. Many publications have focused on predicting operation conditions based on the operation mode for membrane processes ranging from reverse osmosis [54] to ultraﬁltration [55]. Another strategy in predictive modelling of membrane op- eration is to couple physical-based mechanistic models and ANN modelling capabilities to form hybrid models [56]. Using ANNs, there have been attempts to predict the salt retention based on the physical properties of ions [57], to model the rejection of salts [58], and to model the rejection of organic compounds [59] by ANNs. With regard to optimization of membrane fabrication, ANNs can be embedded in optimization problems: Madaeni et al. in 2010 [60] maximized the ﬂux through a membrane using a genetic algorithm (GA). However, GAs cannot guarantee global optimally in ﬁnite time. Furthermore, the previous work performed a single objective optimization which does not account for the inherent trade-o between retention and permeability. Thus, the multi-objective optimization of retention and permeability is desired. The solution of a multi-objective optimization problem is a Pareto front composed of non-dominated points where one objective cannot be improved without having a negative e ect on the other. In this work, we utilize recent theoretical developments for the e cient deterministic global optimization with ANNs embedded [49] using the MAiNGO optimization solver [61], extend the method to multi- objective optimization and apply it to the proposed ANNs to obtain the Pareto front (or upper bound) of retention and permeability. Such deterministic global multi-objective optimization has not been reported in inverse problems optimization for materials systems. To our knowledge, there have been no attempts to identify and optimize the membrane fabrication conditions for nanoﬁltration membranes based on multiple objective such as ion rejections and water ﬂux simultaneously. 5 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 6 3. Methodology 3.1. Fabrication of layer-by-layer (LbL) nanoﬁltration membranes In the scope of this manuscript, all used data sets and the additionally manufactured LbL based nanocomposite nanoﬁltration membranes were fabricated by the dynamic coating method by Menne et al. in 2016 [13] in dead-end mode with constant ﬂux. The applied polyelectrolyte bi-layer consist of the polycation polydiallyldimethylammo- nium chloride (PDADMAC) with a molecular weight of 350 kDa and the polyanion polystyrene sulfonate (PSS) with a molecular weight of 1000 kDa, purchased from Sigma Aldrich and Kemira. The PDADMAC /PSS layers were coated in an alternating fashion. Menne et al.in 2016 [13] investigated the inﬂuence of four aforementioned mem- brane production condition. These parameter consist of the NaCl concentration cNaCl in the coating solution which determines the shape of the polyelectrolyte, the number of polyelectrolyte bilayers Nlayer, the deposition ﬂux Jcoating and the deposition time tcoating applied. The product of the deposition ﬂux Jcoating , the deposition time tcoating and polyelectrolyte concentration of the coating solution applied corresponds to the deposited polyelectrolyte mass m2 PE. The polyelectrolyte concentration of the coating solution was held constant at 1 g L1and not varied in the scope of this and previous manuscripts. The geometry of the tubular support membranes, as well as the module design was kept constant for all data sets. Ten polyethersulfone-based ultraﬁltration hollow ﬁbres provided by Pentair X-Flow (Netherlands) with a length of 400 mmeach and an inner diameter of 0 :8mmwere assembled in a module housing. The hollow ﬁbres feature a pure water permeability of 120 LMH bar1and a separation skin on the lumen side with pores of about 10 nm. In the study presented by Menne et al. in 2016 [13], the four membrane production conditions were varied as shown in Table 1, leading to a large number of di erent combination possibilities. In essence of practi- cability, 63 di erent combinations were fabricated into a module. Their module and thereby membrane performance was characterized by measuring the retention of polyethylenglycol (PEG) with various molecular weights (200, 300, 400, 600, 1000, 1500, 2000, 4000 Da) at a concentration of 1 g L1each, for the molecular weight cut-o (MWCO) measurements. After 4 hours of continuous ﬁltration, samples were analyzed using size exclusion chromatography (SEC) with water as eluent. The salt retention Rwas determined using single salt solutions with a feed concentration of 5mol m3of the salts MgSO4, Na2SO4, MgCl2, NaCl. The measuring process was carried out in cross-ﬂow oper- ation with a constant cross-ﬂow rate at Reynolds numbers of Ref ibret3000 and an absolute feed pressure of 4 :8bar and an absolute retentate pressure of 3 :5bar. The pure water permeability Pwas determined by the permeate ﬂow and the pressure gradient applied. 6 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 7 Table 1. Inﬂuencing membrane fabrication conditions for LbL nanoﬁltration membranes produced via dead end ﬁltration process. A full table of the parameter used can be found in the supplementary material. Parameter Unit Parameter space Box constraints Nbilayerrs 1; 2; 3; 4; 5; 6; 8; 10 1 ¤Nbilayer¤10 tcoatingrmins 2.5; 3.75; 5; 7.5; 10 2 :5¤tcoating¤10 JcoatingrLMHs0; 10; 15; 20; 30; 40 0 ¤Jcoating¤30 m2 PErgm2sJcoatingtcoatingcPE 0¤m2 PE¤5 cNaClrgL1s 0; 0.1; 0.25; 0.5 0 ¤cNaCl¤0:5 3.2. ANN modelling of LbL-nanoﬁltration membranes In this work, the multi-layer perception (MLP) neural network architecture with one hidden layer was chosen for the modelling of LbL-nanoﬁltration membranes. In the hidden layer, a hyperbolic tangent transfer function was applied. 3.2.1. Preparation of the data First, the considered data sets were adapted due to the mixed data types. The sodium chloride concentration cNaCl, deposition ﬂux Jcoating , apparent pore radius rp, apparent charge density Xiand deposition time tcoating are continuous variables as opposed to the number of layers Nlayershowing discrete behavior which is only physically meaningful for integer values. Therefore, the number of layers was transformed into di erent binary classiﬁcations (1 to 10 without 7 and 9). These binary classiﬁcations were set to 1 for the respective number of layers and 0 otherwise. All input and output layers were linearly scaled to normalize all nodes in the same range. Thus, by minimizing the MS E , all parameters are ﬁtted evenly. The neural networks can be trained either for one or for more output entities. 3.2.2. Number of neurons When ﬁtting ANNs to data, a suitable network size, i.e., number of hidden units is important because large ANNs can cause the undesired e ect of over-ﬁtting. This means that large ANNs tend to memorize e ects instead of learning the physical dependencies at hand. Vice versa, too small ANNs can cause under-ﬁtting. A suitable number of neurons in the hidden layer was determined by k-fold cross-validation [62]. Herein, the available data is split into ksubsets. Then, k1 subsets are used for training of an ANN that is subsequently tested on the remaining subset. This process is repeated ktimes, ensuring every subset is used once as testing set. The k-fold 7 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 8 cross-validation was repeated for 1 to 50 hidden units with k9 subsets. The number of hidden units leading to the minimal mean squared error ( MS E ) on all test sets on average is set. 3.2.3. Training After determining a suitable number of hidden neurons, the ANNs are trained. As the targeted accuracy of the ANNs should be in the range of the measuring inaccuracy and the available data is limited, the experimental data set was expanded by the expected value the measurement error of each data point. Consequently, the output vector was extended to its threefold size. The each di erent erroneous measurements of every data point are accounted for by expanding around the expected value. The resulting data was split into 70% training, 15% test and 15% validation set at random. Then, the training algorithm (for the synthesis of the membrane fabrication conditions: [63] and else: [64]) was used to minimize the MS E between on the training set where di erent outputs were weighted by their reciprocal percentile error. In order to prevent overﬁtting, early stopping was used where intermediate training results are checked on the validation set. The training is terminated when error of the validation data increased six times in a row. Afterwards, the resulting network was tested using the testing data. This procedure was repeated ten-times with hidden unit numbers in the range of the optimal number of hidden units 3 and the ANN with the lowest MS E on the test set was chosen. 3.2.4. The three networks In the scope of this manuscript three MLP networks are presented. The ﬁrst MLP relates the membrane fabrica- tion parameter ( cNaCl,Jcoating ,tcoating ,Nlayer) to the membrane performance parameter. Accordingly, the input vector ex- panded from its originally 4 di erent parameter ( cNaCl,Jcoating ,tcoating ,Nlayer) to 11 di erent parameter ( cNaCl,Jcoating ,tcoating , 8 di erent numbers of layers). The second MLP uses the dependency of the deposition ﬂux and time span, therefore reducing the input to 3 dif- ferent membrane fabrication parameter ( cNaCl,m2 PE,Nlayer) which are related to the membrane performance parameter. Analogously, the fabrication input vector reduced to 3 (10) di erent input conditions. The third MLP correlates the 3 di erent membrane fabrication parameter ( cNaCl,m2 PE,Nlayer) to the apparent pore radius rpand apparent charge density Xi. Since the measuring inaccuracies are not known, the MLP network was trained and optimized di erently. Due to the scarce data set, the data was divided into 70 % training, 30 % validation and additionally 0 % test set at random. The number of iterations during training was limited to prevent overshooting between the training nodes. The training process was repeated tenfold with hidden units in the range of the previously determined optimal number of hidden units. 8 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 9 3.3. Transport model of LbL-nanoﬁltration membranes The separation of ions by nanoﬁltration is described by three distinct transport domains: through, at, and towards the membrane. The problem set was simpliﬁed to one dimension, only describing the changes in orthogonal direction at turbulent feed ﬂow conditions of the membranes in a module housing. The problem at hand demands a transport model that not only describes the transport, but also quantiﬁes the membrane premised on apparent model-based properties. First, for the transport through the membrane an Extended Nernst-Planck equation based model was selected, because it can describe the transport of ions with regards to the layer build-up on a porous support. Hence, in the scope of this manuscript the two-parameter model proposed by Bowen et al. in 2002 [65] was implemented. The modelling of solute transport inside nanoﬁltration membranes depends on whether the solute is uncharged or charged, due to the strong inﬂuence of electric e ects in nanoﬁltration membranes. As proposed by Bowen et al. in 2002 [65], the data ﬁtting for the transport model is processed in two parts - an uncharged and a charged ﬁt. The measurements of uncharged solute retention are used to ﬁt the theoretical pore radius rp. The apparent charge density Xiis determined using charged solute retention measurements for each salt separately. Second, partitioning e ects arising from the transition from the bulk to the pore phase were accounted for by introducing a distribution coe cient. In the scope of this work, the steric exclusion as well as the electric exclusion stemming from the Donnon-potential were used to calculate the distribution coe cient. Third, the transport model must account for the concentration polarization towards the separation layer. This e ect considers that the mass transport resistances build-up at the laminar boundary layer. A detailed description of the model and its assumptions can be found in the supplementary material. 3.4. ANN aided selection of membranes with desired membrane performance conditions For the ANN aided selection of membranes with desired membrane production condition the ANN was trained and solved using a local solver [66]. The local solver was initiated by manually selected points from the training set and the progress of the solver, i.e., its iterations were observed. This method allows to also utilizing (sub-optimal) intermediate results that are similar to existing membranes reducing the risk of false predictions far away from training data. The distances of the membrane fabrication parameter for the presented membranes were observed by formation of the weighted euclidean distance of the input variables by Equation (1). jmin i¸ jujuExp;j;i uExp;j;max 2 (1) wherecorresponds to the calculated distance to a training point. irefers to the corresponding initial experi- ment (training set), jrefers to the corresponding component of vector and ajis a weight that scales the distances in component j. The weight scales were set to the maximum value respectively. 9 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 10 3.5. Deterministic global multi-objective optimization For determining the best possible membranes that can be produced, a multi-objective optimization is set up and solved globally. The optimization problem is formulated as follows: max u Ppurewaterpuq Ripuq  w1;MLPpuq w2;MLPpuq s.t. min i¸ jujuExp;j;i aj 2 ¤ ulb¤u¤uub(2) whereis the maximum allowed distance to a training point with the constants .irefers to the corresponding initial experiment (training set), jrefers to the corresponding component of vector and ajis a weight that scales the distances in component j. Further, the functions w1;MLPandw2;MLPdescribe the pure water permeability Pand retention Riof ionipredicted by the MLPs. Box constraints are listed in Table 1. There exist various solution strategies for multi-objective optimization problems, e.g., -constraint method and Bayesian approaches [67]. In this work, the proposed multi-objective optimization problem was reformulated to a set of single-objective optimization problems using the -constraint method. This method allows us to utilize the recently developed and e cient method for global deterministic optimization of problems with ANNs embedded [49]. This method formulates the ANNs in a reduced-space, which reduces problem size and accelerates global optimization. Further, the framework includes tailor-made relaxations of ANNs and utilizes the MAiNGO optimization solver [61]. 10 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 11 4. Results 4.1. ANN training of membrane fabrication conditions The four di erent membrane fabrication parameters, sodium chloride concentration cNaCl in the polyelectrolyte solution, deposition ﬂux Jcoating , deposition time tcoating , and the number of layers Nlayerapplied were set as input vari- ables of the ANN. The ionic retention Rof the salts MgSO4, Na2SO4, MgCl2, NaCl, and the pure water permeability Pwere set as output variables of the ANN. The ANN was trained for the synthesis of the membrane production con- ditions and the corresponding membrane transport properties (cf. Figure 1 (ANN Model)). A k-fold cross-validation was used to train an ANN with 32 hidden neurons in order to avoid overﬁtting or underﬁtting (error in the range of the measuring inaccuracies). In this case, the standard deviation was used as an additional qualitative measure. The quality of the resulting ANN and statistical evaluation is depicted for MgSO4and MgCl2in Figure 2 A. The training-set evaluation proofs the ability to train an ANN from the available experimental data, whereas the test-set shown in Figure 2 B provides additional information about the predictivity towards data that was not used for training and validation. The low standard deviation of 1.5% for MgSO4and 3.3% for MgCl2show a high correlation with supported observation through the test set. The regression plots for Na2SO4, NaCl and the pure water permeability show comparable behavior and can be found in the supplementary material. Figure 2. Ability to train the ANN Model for the synthesis of the membrane fabrication conditions showing A) the regression of the ANN training with four inputs indicating a good ability to train the ANN. Depicted is the predicted retention of the ANN versus the experimental measured retention of MgSO4and MgCl2. In B) an additional test set is positively validated by the ANN. The data shown in B) was not used for training and validation of the ANN. 11 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 12 4.2. ANN model for the synthesis of the membrane fabrication conditions After successful training, we now demonstrate that desired design speciﬁcation for the separation characteristics could be obtained based on the available ANN. Therefore, a local optimization solver is used to maximize the pure water permeability that is predicted by the ANN. The local solver is initialized at membrane fabrication parameters of an existing membrane. Thus, the approach is able to improve existing membranes in a local neighbourhood. Subsequently, the proposed membranes were fabricated and characterized. A target area was deﬁned for one salt each as depicted in Figure 3 A-B. A minimum of 80% salt retention should to be obtained for MgSO4in Figure 3 A and for MgCl2in Figure 3 B, respectively. Not just any membrane should be found, but the one with the highest trusted pure water permeability. Comparing the regression, the predicted retention precisely matched the measured retention. These results illustrate for the ﬁrst time that an ANN is able to identify design parameters for ion separation membranes manufactured using a LbL-based synthesis method. It should be noted that increasing the pure water permeability in the trade-o relation to any extent would lead to non-achievable ion retention. This was observed by formation of the weighted euclidean distance of the input variables (cf. methods section Equation (1)). For the presented membranes the weighted euclidean distance distances from the known empirical data points were calculated toMgS O 40:03279 and MgCl 20:00742, respectively. By diverting to far away from known data points, the predictive accuracy of the ANN decreases. This e ect is shown by way of example in the supplementary material. Figure 3. Synthesis of the membrane fabrication conditions by the ANN model based on the desired membrane performance using an using single objective optimisation with ANN-embedded and local solver optimisation framework showing A) an optimisation yielding a membrane with MgSO4retention 80% or higher and the highest pure water permeability possible at MgS O 40:03279 and showing B) an optimisation yielding a membrane with MgCl2retention 80% or higher and the highest pure water permeability possible at MgCl 20:00742. 12 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 13 4.3. Reduction of the ANN complexity To further reduce the complexity of the ANN, we aggregated two inputs of the ANN, deposition time tcoating and ﬂux Jcoating , to deposited polyelectrolyte mass m2 PEby multiplication. This less complex ANN uses the three di erent membrane fabrication conditions, the sodium chloride concentration cNaCl in the polyelectrolyte solution, the deposited polyelectrolyte mass m2 PE, and the number of layers Nbilayerapplied as input variables. The retention Rof the salts MgSO4, Na2SO4, MgCl2, NaCl, and the pure water permeability Premain as output variables. This reduction of the network complexity yielded the lowest mean squared error (MSE) employing an ANN with 25 hidden neurons through a k-fold cross-validation without overﬁtting nor underﬁtting. The regression shown in Figure 4 A show high correlation of the set parameters quantiﬁed by a low standard deviation of 1.6% and 3.2% for MgSO4 and MgCl2, respectively. The performance of the ANN with three and four inputs is comparable, leading to the conclusion that three inputs are su cient to quantify the inﬂuence of the membrane fabrication conditions on the membrane performance. The regression plots for Na2SO4, NaCl and the pure water permeability show comparable behavior and can be found in the supplementary material. 4.4. ANN model for calculation of the upper bound for the pure water permeability and retention trade-o By using the local optimization ANN aided selection, an improved membrane close to existing data points has been identiﬁed above. As the solution only leads to a local optimum around a given initial membrane, the question raises whether a global optimum can be identiﬁed. The second question to be answered is whether a Pareto front (or upper bound for the pure water permeability and retention trade-o ) can be identiﬁed for the considered membrane fabrication conditions synthesis method. For this, the multi-objective optimization problem was reformulated to a set of single-objective optimization problems using the -constraint method and was solved using the recently proposed deterministic global solution approach for problems with ANNs embedded [49]. For the optimization the MAiNGO optimization solver was used [61]. We hypothesize that this solution strategy enables the identiﬁcation of the inherent trade-o between ionic retention and pure water permeability. The solution will further provide the respective membrane production condition for the membrane with the highest salt retention Rfor a given pure water permeability P. The Pareto front obtained for MgSO4retention and pure water ﬂux and the initial measured data points that were used for training of the ANN are shown in Figure 4 B. The discontinuity of the Pareto front (jumps) can be explained by the discrete layer selection and its predominant inﬂuence on the membranes performance. The optimization algorithm can divert from the known data points by the means of a weighted euclidean distance . This allows the Pareto front to shift towards higher salt retention Rat higher pure water permeabilities P. When the optimization framework is allowed to divert too far from the membrane fabrication parameter used for training, the identiﬁed upper bound shifts to nonphysical results. This phenomenon can be observed in examples given in the supplementary material where a shift of the Pareto front to an unfeasible retention above 100% can be observed. 13 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 14 The number of training sets available for the training of the ANN has a signiﬁcant inﬂuence on the reliability of the computed Pareto front. Increasing the number of training sets will obviously lead to more accurate predictions and a more reliable Pareto front. In the supplementary material, however, we show that even with a small number of training data points a reasonable good prediction towards the desired membrane properties can be made. The optimized membrane based on the initial least square solver optimization from Figure 3 A is displayed as well in Figure 4 B. The optimum given by the global deterministic multi-objective method indicates even further improvement potential over the local single objective method for both retention and permeability. Figure 4. Pareto front optimisation using an ANN embedded global deterministic multi-objective optimisation framework. A) Regression of the reduced ANN training with three inputs showing the predicted versus the experimental retention of MgSO4and MgCl2. B) Developed Pareto front for the retention of MgSO4versus the pure water permeability with comparison of the membrane found by local solver optimisation. 4.5. Combination of ANN model with transport model to create insight to the delicate layer build up Above, we used the ANN-based optimization to identify the upper bound trade-o between ion retention and solvent permeability for the fabrication of LbL nanoﬁltration membranes given a certain polyelectrolyte system. We examined and modelled the fabrication parameter space based on measurements performed when applying a constant trans-membrane ﬂux. However, it is well known that retention properties can vary with increasing transmembrane ﬂux: the retention of ions increases initially as solvent convection dominates over ion di usion at increasing driving force, reaches a maximum, and decays with further increased ﬂux due to concentration polarization. This complex behavior and interplay between solvent and ion ﬂuxes is currently still debated, yet it is generally accepted that the physico-chemical features are well described by the extended Nernst-Planck Equation [38, 65]. Hybrid models o er the possibility to combine physical models with the ANN models. Combining of a physical model with an ANN enables to extrapolate beyond the experimental data set. We now incorporate an extended Nernst- 14 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 15 Planck Model to model the dependence of mass transport rates [65] and use the ANN to model the input parameters to the physical model. In fact, this methodology now also provides for the ﬁrst time insight to what extend synthesis parameters inﬂuence physical quantities utilized as input parameters to the model. For the purpose of demonstration, we assume the transport of charged species through the LbL-nanoﬁltration membrane to be su ciently characterized by the model system proposed by Bowen et al. in 2002 [65]. Only an apparent pore radius rpand an apparent space charge density Xi- speciﬁc for each salt - is required to quantify the nonlinear transport behavior. However, next to ﬂux and ion retention data, the transport model ﬁt also requires the retention data of uncharged species of di erent size, i.e., molecular weight in order to quantify an apparent pore radius rp. The experimental data set for the 63 modules was therefore extended by polyethylenglycol (PEG) retention measurements, also known as molecular weight cut-o measurements. Based on the ion retention data, a charged species model ﬁt yields the apparent space charge density Xifor each salt separately. This salt dependency actually represents the fact that the ionic nature of the feed solution inﬂuences the properties of the separation layer; the latter adapts to the feed environment. The quality of the transport model is depicted for MgSO4in Figure 5 A and shows that the model can represent the salt retention well over a wide range of values ranging between 10% to close to 100%. The regression plots for MgCl2, Na2SO4, NaCl can be found in the supplementary material. The resulting nonlinear relationship between the retention and the transmembrane ﬂux is displayed for one selected membrane as shown in Figure 5 B. Now, the correlation between the model parameters for the apparent pore radius rpand the salt-speciﬁc apparent space charge density Xican be learned based on the membrane fabrication conditions (cf. Figure 1 C) by a new ANN. This training procedure followed the same input parameter space as the second ANN, the 3 di erent membrane production conditions being (1) sodium chloride concentration cNaClin the polyelectrolyte solution, (2) the deposited polyelectrolyte mass m2 PE, and (3) the number of layers Nlayerapplied. The apparent pore radius rpand the apparent space charge density Xifor each salt iwere set as output entities. The k-fold cross-validation yielded the lowest mean squared error employing an ANN with 35 hidden neurons. In Figure 6 both the parameters, the apparent pore radius rpand the salt-speciﬁc apparent space charge density Xishow good agreement between the simulated value obtained for each membrane by the extended Nernst-Planck Model and its predicted value by the ANN. The apparent pore radius varies over a range of 0 :5nmto about 2 nm. The apparent space charge varies over a range of 20mol m3 to300mol m3. These results suggest a high correlation quantiﬁed by a low standard deviation of 15 :6mol m3 and 0:1nmfor the apparent space charge density XMgS O 4of MgSO4and the apparent pore radius rp, respectively. The regression plots for the apparent space charge density Xiof the salts MgCl2, Na2SO4, and NaCl can be found in the supplementary material. With this new ANN, it is possible for the ﬁrst time to predict membrane fabrication conditions if retentions are required to be obtained at a given transmembrane ﬂux level. 15 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 16 The shown hybrid model cannot only be used for synthesis of the desired membrane fabrication conditions, but also presents a methodology to generate physical insight into the build-up of the separation layer. The model param- eters linked to an ANN can provide information about the physical e ects based on membrane fabrication conditions applied. In Figure 7 the impact of the variation of the ionic strength in the coating solution on the apparent space charge density XMgS O 4and apparent pore radius rpfor constant 5 gm2deposited polyelectrolyte and 1 bi-layer are investigated by the means of the trained ANN. In Figure 7 A the ANN predicts an increasing space charge density with an increasing ionic strength in the coating solution by varying the NaCl concentration. The observed trajectory is supported by recent investigations. The layer thickness increases with the amount of salt added to the coating solution coupled with more adsorbed mass of polyelectrolyte. The charges of the poly-cations and poly-anions compensate partly and form ion bonding known as intrinsic charge compensation. The rest of the polyelectrolyte charges compensate extrinsically by the counter ions. The higher the concentration of counter ions in the coating solution the more extrinsic compensated charges can be found in the multi-layers [68, 69]. These e ects lead to a higher charge density as predicted by the ANN. In Figure 7 B the ANN predicts an decreasing apparent pore radius rpwith an increasing ionic strength in the coating solution. The predicted trajectory can be explained by recent ﬁndings regarding the two stages of the layer build-up. First, the pores of the support membrane become smaller through the thickness of the adsorbed polyelec- trolyte layer. Second, by adding more layers, the pores of the support are covered completely. This results in a membrane characteristics mainly depended on the polyelectrolyte multi-layer. The thickness of a layer depends on the NaCl concentration in the coating solution, leading to thicker layers due to charge screening and compensation e ects as compared to coatings without salt. Without salt in the coating solution, the charged groups in the polyelec- trolyte chain repel each other, forming an extended chain. By adding salt, the excess counter ions screen the charges of the polyelectrolyte chain resulting in a fuzzy layer assembly [70, 68, 69]. Higher NaCl concentrations in the coating solution lead the forming of thicker layers, which decrease the pore radius, when adsorbed on the pores surface. The trained ANN correctly predicts this e ect. 16 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 17 Figure 5. Transport model ﬁt showing A) the regression of the model ﬁt. The regression is shown for the simulated versus the experimental retention of MgSO4and presents the good quality of the ﬁt. Based on the ﬁtted model based parameter the B) trajectory of simulated retention of MgSO4, MgCl2, MgSO4, and MgSO4can be obtained dependent on the transmembrane ﬂux applied. Additionally the experimental data points of the ﬁt are displayed. 17 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 18 Figure 6. Combining the transport model with the ANN into a hybrid model to synthesise the desired membrane based on process operating conditions at hand. Showing A) the regression of the reduced ANN training with three inputs by the predicted apparent space charge density XMgS O 4of MgSO4versus the simulated apparent space charge density XMgS O 4of MgSO4obtained by the transport model and B) the regression of the predicted apparent pore radius rpversus the simulated apparent pore radius rpobtained by the transport model. 18 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 19 Figure 7. Hybrid model to generate physical insight into the build-up of the separation layer. A) The impact of the variation of the ionic strength in the coating solution on the apparent space charge density XMgS O 4for constant 5 gm2deposited polyelectrolyte and 1 bi-layer. B) The impact of the variation of the ionic strength in the coating solution on the apparent pore radius rpfor constant 5 gm2deposited polyelectrolyte and 1 bi-layer. 19 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 20 5. Conclusion To this day, it remained a challenge to correlate the performance of LbL-nanoﬁltration membranes to its chemistry and structure. With this study, we present a novel approach using ANNs to link the mass transport properties of LbL- nanoﬁltration membranes to synthesize membrane fabrication conditions. An ANN-based optimization framework identiﬁes unique membrane fabrication conditions with maximized ion rejection characteristics and permeability. Moreover, by an extension of the framework to an ANN-based global deterministic multi-objective optimization we did identify the upper bound of the delicate ion rejection characteristics and permeability trade-o . This two method- ologies yield tailored optimal membrane fabrication conditions. Consequently, we present a hybrid model consisting of a state-of-the-art transport model to depict the transport of charged species through the LbL-nanoﬁltration mem- brane in combination with an ANN. The proposed methodology now enables the design of membrane separation properties tailored to the application. Also, it generates physical insight by linking the apparent model-based mem- brane properties to the membrane fabrication conditions. The versatility and the accuracy of the methodology suggests that a future platform for membrane synthesis should rely on a data driven methodology of a synthetic parameter space that can be speciﬁcally tuned over a wide range. ANNs can be systematically used in the development phase, as well as for systematic investigation of physical properties of existing membranes. Acknowledgement M.W. acknowledges the support through an Alexander-von-Humboldt Professorship. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and inno- vation program (grant agreement no. 694946), and German Federal Ministry of Education and Research (BMBF) under the project UO-Rohrfabrikation (03XP0100E) and the project E ueNF (02WIL1486). A.S. and A.M. grate- fully acknowledge the ﬁnancial support of the Kopernikus-project SynErgie by the Federal Ministry of Education and Research (BMBF). They would like to thank the project supervision by the project management organization Projekttr ¨ager J ¨ulich (PtJ). Author contributions D.R., D.M. and M.W. formulated the hypothesis to developed artiﬁcial neural networks to link the mass transport properties of LbL-nanoﬁltration membranes to synthesize membrane fabrication conditions. D.M. and J.K. provided the measurements of the fabricated membranes. A.S. and A.M. developed the methodology to obtain the Pareto front and the framework for global deterministic optimization. D.R. and A.S. developed the Pareto Front optimization for this manuscript. D.R. and L.v.K. implemented the model-based predictions based on models found in literature. D.R. and J.K. investigated the model-based properties. D.R., A.S., A.M. and M.W. wrote the manuscript. All authors gave ﬁnal approval of the manuscript to be published. 20 D. Rall et al. /Journal of Membrane Science 00 (2018) 1–24 21