dataesr/scientific-paragraphs-categorization huggingface dataset:
It is based on a scoring function assigning a score to every pair consisting of a database and a potential output, and it attempts to return an output with approximately maximal score, given the input database. As shown by Gaboardi et al. [13] , the exponential mechanism can be added as a primitive to Fuzz with type: expmech : ! ∞ set(O) ⊸ ! ∞ (! ∞ O ⊸! s dB ⊸ R) ⊸! ϵ dB ⊸ ⃝ P O, where O is the type of outputs. The function LpMech is an instance of the exponential mechanism where O is ⊗ d p R and the score is λyλD. ||f (D) -y|| p. To define the L p mechanism with this recipe, we need to reason about the sensitivity of this scoring function. In Fuzz, this would not be possible, since the language does not support reasoning about the sensitivity of f measured in the L p distance. In Bunched Fuzz, however, this can be done easily.
That is, the goal is to minimize the loss function L(θ, (x, y)) = 1 n • Σ n i=1 (⟨x i , θ⟩ -y i ) 2. The algorithm starts with an initial parameter vector (0,. , 0) and it iteratively produces successive θ vectors until a termination condition is reached. The Fuzz program uses the data-type bag τ representing bags or multisets over τ. A bagmap primitive is given for it. The type I is the unit interval [0, 1]. The main function is called updateP arameter and updates one component of the model θ; it is computed in the following way: with the function calcGrad : db → R, compute a component (∇L(θ, (x, y))) j of the R d vector ∇L(θ, (x, y))  9. then Laplacian noise is postcomposed with calcGrad in the updateP arameter function. This uses a privacy budget of 2ϵ. It has to be done for each one of the d components of ∇L(θ, (x, y)), thus on the whole, for one step, a privacy budget of 2dϵ.
-The iterative procedure of gradient descent is given by the function gradient in Fig. 8  p. 19 of  [25]. We forget here about the adaptative aspect and just consider iteration with a given number n of steps. In this case by applying n times updateP arameter one gets a privacy budget of 2dnϵ. We modify the program as follows to check it in Bunched Fuzz and use the L p -mechanism. Instead of computing over R we want to compute over ⊗ d p R for a given p ≥ 1, so R d equipped with L p distance. The records x i are in ⊗ d p I and the labels y i in I. The database type is dB = bag (I ⊗ p (⊗ d p I)). The distance between two bags in dB is the number of elements by which they differ. We assume a primitive bagV ectorSum with type ! d 1/p bag (⊗ d p I) ⊸ ⊗ d p R (it could be defined as the vectorSum defined above for sets, using a sum primitive for bags).
Given a bag m, (bagV ectorSum m) returns the vectorial sum of all elements of m. We can check that the sensitivity of bagV ectorSum is indeed d 1/p because given two bags m and m ′ that are at distance 1, if we denote by u the vector by which they differ, we have: d (⊗ d p R) (bagV ectorSum(m), bagV ectorSum(m ′ )) = ||u|| p ≤ (Σ d j=1 1) 1/p = d 1/p By adapting the calcGrad Fuzz term of  [25]  using bagV ectorSum we obtain a term V ectcalcGrad with the Bunched Fuzz type ! ∞ ⊗ d p R ⊸! d 1/p db ⊸ ⊗ d p R. Given a vector θ and a database (y, x), V ectcalcGrad computes the updated vector θ ′. Finally we define the term updateV ector by adding noise to V ectcalcGrad using the the L p -mechanism. Recall the type of LpMech: ! ∞ (! s db ⊸ ⊗ d p R) ⊸ ! ϵ db ⊸ ⃝ P (⊗ d p R). We define updateV ector and obtain its type as follows: updateV ector = λθ.
Recent work showed how this could be amended via a path construction that recasts relational program properties as sensitivity properties  [3]. Roughly speaking, instead of working directly with the Hellinger distance d H , the authors consider a family of relations R α = {(µ 1 , µ 2 ) | d H (µ 1 , µ 2 ) ≤ α}. Such a relation induces another metric on distributions, d α,H , where the distance between two distributions is the length of the shortest path connecting them in the graph corresponding to R α. This allows them to express the composition principles of the Hellinger distance directly in the Fuzz type system, albeit at a cost: the type constructor for probability distributions is graded by the distance bound α. Thus, the sensitivity information of a randomized algorithm with respect to the Hellinger distance must also be encoded in the codomain of the function, as opposed to using just its domain, as done for the original privacy metric of Fuzz.
By contrast, Bunched Fuzz does not require the grading α be-cause it can express the composition principle of the Hellinger distance directly, thanks to the use of the L 2 distance on bunches. Duet  [20]  can be seen as an extension of Fuzz to deal with more general privacy distances. It consists of a two-layer language: a sensitivity language and a privacy language. The sensitivity language is very similar to Fuzz. However, it also contains some basic primitives to manage vectors and matrices. As in Fuzz, the vector types come with multiple distances but differently from Fuzz, Duet also uses the L 2 distance. The main reason for this is that Duet also supports the Gaussian mechanism which calibrates the noise to the L 2 sensitivity of the function. Our work is inspired by this aspect of Duet, but it goes beyond it by giving a logical foundation to L p vector distances.
Conclusion and Future work In this work we have introduced Bunched Fuzz, a type system for reasoning about program sensitivity in the style of Fuzz  [23]. Bunched Fuzz extends the type theory of Fuzz by considering new type constructors for L p distances and bunches to manage different products in typing environments. We have shown how this type system supports reasoning about both deterministic and probabilistic programs. There are at least two directions that we would like to explore in future works. On the one hand, we would like to understand if the typing rules we introduced here could be of more general use in the setting of probabilistic programs. We have already discussed the usefulness for other directions in the deterministic case  [19]. One way to approach this problem could be by looking at the family of products recently identified in  [5].
Given a dataset {x i , y i , i = 0,. , n}, the Hilbert kernel estimator f (x) of the regression function f (x) (see Methods for problem setup and notation definitions) is a Nadaraya-Watson style estimator employing a singular kernel, f (x) = n i=0 w i (x)y i , (1) w i (x) = ∥x -x i ∥ -d n j=0 ∥x -x j ∥ -d , (2) where w i (x) is the weight of the i-th data in the prediction f (x), for a given ddimensional input x. Note that when x = x i is an input vector from the dataset, w i (x = x i ) = 1, w j (x = x i ) = 0, for all j ̸ = i, and the prediction exactly coincides with the dataset result, f (x = x i ) = y i (true interpolation). We show in Fig. 1a  an example of the Hilbert kernel regression estimator in one dimension, d = 1 (an application to classification by thresholding a soft classifier is shown in Extended Data Fig.
94, and its main oscillation. We illustrate these results in SI Movie S1, which shows a 2-minute collective dynamics of 2 Hilbert agents. We have also studied the impact of the memory M. We find that using no memory (M = 0) for the Hilbert input and data vectors of Eqs. (11, 12)  would not produce coordinated trajectories. Our analysis below of the dynamics without the rejection procedure enforcing the presence of the wall will confirm that a memory M = 0 fails to reproduce the correct fish behavior. We found that M = 1, M = 2, and M = 3 lead to comparable results, while the agreement with experiments is degrading for a memory in the range M = 4-10. A possible reason could be the increase of the dimension d of the input and data vectors with M (see Eq. (  13 )), resulting in a scarcity of data in the d-dimensional configuration space (d = 16 for M = 2; d = 48 for M = 10).
, t -M ∆t. Indeed, for M > 0, we find that the Hilbert agents remain cohesive, stay inside the tank for the vast majority of the simulations (87 % for M = 2), and only make brief and small (1. 3 cm away from the tank wall, on average) excursions outside the tank. This is illustrated for M = 2 in the Extended Data Fig. 2 , which corresponds to Fig. 3  but without the rejection procedure. The deep learning model of  [26]  presents a similar stability without the rejection procedure, showing that both Hilbert (for M > 0) and ML models can implicitly capture and learn the presence of the wall from the data. In fact, note that the deep learning model of  [26]  also exploits a memory of the past, encoded in the input data (with an effective M = 5) and in its architecture involving LSTM layers. SI Movie S2 shows a 2-minute collective dynamics of 2 Hilbert agents without enforcing the presence of the wall, confirming that the Hilbert interpolation scheme has implicitly learned the presence of the wall from the data.
On the other hand, both the ANN-based and Hilbert generative models are purely data-driven and neither offer a similar degree of interpretability as mathematical models. However, we posit that the Hilbert scheme nevertheless offers significant advantages over the ANN-based model. The Hilbert generative model leads to a straightforward implementation, consisting of a few lines of code (see Fig. 2 ). There is no training, parameter tuning, or model selection, and it offers strict reproducibility, portability, and transparency, which is particularly important for scientific applications. This framework also permits the reuse of generated data in real time to augment the experimental dataset. Moreover, the relative contribution of training data points contributing to a specific prediction is transparently characterized using the corresponding weights, and the procedure flexibly adapts those weights across training data points for any given prediction.
Fish trajectory training data set details In this work, we have exploited the fish data published in  [25, 60]. The fish are swimming in shallow water, and only their position and heading in the horizontal plane of the tank have been recorded so that the system is effectively 2-dimensional. All trajectories have been subsampled from the original 30 FPS, resulting in a considered time-step of ∆t = 0. 1 s. In addition, a Gaussian smoothing with a window of 0. 15 s has been applied to the trajectories, in order to reduce the noise in the computation of the velocities and accelerations of the fish, using centered difference. Finally, we augmented the data by partly exploiting the rotational symmetry of the system: x → ±x, y → ±x, x ↔ y.
Outputs Inputs Hidden Figure  5 : A multilayer perceptron model (here with only one hidden layer, but there can be many more). the statistical theory of learning (in particular the works of Vladimir Vapnik  [22] ). A landmark algorithm developed at that time was the support-vector machine (SVM)  [23]  which worked well with small training data sets and could handle non-linearities through the use of kernels. The machine learning field continued to expand through the 2000s and 2010s, with new approaches but also more mature software packages such as scikit-learn  [24]. More generally, it is actually important to have in mind that what is currently called AI owes more to statistics (and other mathematical fields such as optimization in particular) than to modeling of brain circuitry and that even approaches that take inspiration from neurobiology can actually be viewed as complex statistical machineries.
2012 saw the revival of neural networks and the beginning of the era of deep learning. It was undoubtedly propelled by the considerable improvement obtained on the ImageNet recognition challenge which contains 14 million natural images belonging to 20,000 categories. The solution, proposed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton  [25] , was a convolutional neural network with a large number of layers, hence the term deep learning. The building blocks of this solution were already present in the 1980s but there was not enough computing power nor large training datasets for them to work properly. In the interval, things had changed. Computers had become exponentially more powerful and, in particular, the use of Graphical Processing Units (GPU) considerably sped up computations.
The expansion of the Internet had provided massive amounts of data of various sorts such as texts and images. In the subsequent years, deep learning  [26]  approaches became increasingly sophisticated. In parallel, efficient and mature software packages including TensorFlow  [27] , PyTorch  [28]  or Keras  [29] , whose development is supported by major companies such as Google and Facebook, enable deep learning to be used more easily by scientists and engineers. Artificial intelligence in medicine as a research field is about 50 years old. In 1975, an expert system, called MYCIN, was proposed to identify bacteria causing various infectious diseases  [30]. More generally, there was a growing interest in expert systems for medical applications. Medical image processing also quickly became a growing field.
Box 1: Definition of machine learning Machine learning definition  (Mitchell, 1997 [31] ): • "a computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance at task T, as measured by P, improves with experience E. " Example: learning to detect tumours from medical images • Task T: detect tumours from medical image • Performance measure P: proportion of tumours correctly identified • Experience E: examining a dataset of medical images where the presence of tumours has been annotated examples for which the output that we are trying to learn is known.
The clusters are not known a priori. Examples 1 and 2 will be referred to as clustering tasks. -Example 3. Given a large set of medical characteristics (various biological measurements, clinical and cognitive tests, medical images), find a small set of variables that best explain the variability of the dataset. This is a dimensionality reduction problem. • Reinforcement learning. Learns by iteratively performing actions to maximize some reward. -Classical approach used for learning to play games (chess, go. ) or in the domain of robotics. -Currently few applications in the domain of brain diseases. Colliot amples of common regression methods include simple or multiple linear regression, penalized regression, and random forest regression. Finally, there are many other tasks that can be framed as a supervised learning problem, including for example data synthesis, image segmentation and many others which will be described in other chapters of this book.
A first reason may be that the training set is too different from the general population (for instance, we have trained a model on a dataset of young males and we would like to apply it to patients of any gender and age). Another reason is that, even if the training set characteristics follow those of the general population, the learned function f may be too specific to the training set. In other words, it has learned the training set "by heart" but has not discovered a more general rule that would work for other examples. This phenomenon is called overfitting and often arises when the dimensionality of the data is too high (there are many variables to represent an input), when the training set is too small or when the function f is too flexible. A way to prevent overfitting will be to modify the cost function so that it not only represents the average error across training samples but also constrains the function f to have some specific properties.
Acknowledgments The author would like to thank  Johann Faouzi  for his insightful comments. This work was supported by the  French government  under management of  Agence Nationale de la Recherche  as part of the " Investissements d'avenir" program , reference  ANR-19-P3IA-0001  ( PRAIRIE 3IA Institute ) and reference  ANR-10-IAIHU-06  ( Institut Hospitalo-Universitaire ICM).
Acknowledgements We thank  Carolina Giorgetti ,  Luca Dal Zilio ,  Giacomo Pozzi ,  Lauro Chiaraluce  and  Marco Maria Scuderi  for insightful discussions. We thank the Editor  Rebecca Bendick  and two anonymous reviewers for their valuable comments. AG and CM were supported by  European Research Council Advance  Grant  835012  ( TEC-TONIC ). MC participated in this work as Principal Investigator of the  European Research Council (ERC)  project  FEAR  (grant  856559 ) under the European Community's  Horizon 2020 Framework Programme. GM acknowledges support from  NUS   Startup Grant   22-3565-A0001-1 , and  MOE Tier 2 Grant 22-5191-A0001-0.
Finally, in Section 5, we give an application to the distribution properties of the continued fraction expansions of elements in F q pY q, thus giving an analogue to the result of Dinaburg-Sinai in  [DS]  described in the beginning of this Introduction. Acknowledgments. The authors warmly thank Amos Nevo for having presented one to the other in a beautiful conference in Goa in February 2016, where the idea of this paper was born. The first author thanks the IHES for two post-doctoral years when most of this paper was discussed, and the Topology team in Orsay for financial support at the final stage. 2 Background on function fields and their modular groups.
Acknowledgements. The work of the first and third authors was supported by the  French ANR   SANGRIA  project ( ANR-21-CE39-0006 ). The work of the second author was in part support by the  Programma ricerca di ateneo UNICT   2020-22 linea 2. The work of the third author was supported by the  French PEPR Cybersecurité   SecureCompute  project ( ANR-22-PECY-0003 ). The work of fifth author was supported by  ERC  ( ERC-2020-COG ).
EPICS is a set of software tools and applications which provide a software infrastructure for use in building distributed control systems to operate devices such as Particle Accelerators, Large Experiments and major Telescopes. Such distributed control systems typically comprise thousands of computers, networked together to allow communication between them and to provide control and feedback of the various parts of the plant. To achieve this goal, each system in EPICS defines a set of control variables, the values of which are published and accessible to all other systems in the network. EPICS calls each of these variables a "Process Variable" (PV). Due to the large size of ITER, it will generate more than one million control signals in its first phase. And for its full operational state, it is estimated at more than three million control signals.
Requirements The main requirements to be met by the EPICS control variable storage system in ITER  [3, 4]  are: -Continuous storage: Data must be able to be stored continuously over time regardless of the pulse number or other parameters associated with the experiment. Archiving services must support storage rotation mechanisms. -Limited latency: Data that is stored must be accessible within a limited amount of time. Any solution that forces the file to "close" before making the data available is not acceptable. -Changes over time: In the case of control variables, some of their properties may change over time. The base element for data storage in ITER is the HDF5 file. Although, this HDF5 file does not need to correspond at the physical storage level to an ". h5" file (as will be seen in point 2.
Alternative implementation At present, there is one implementation "The EPICS Archiver Appliance"  [5] , that has some characteristics that are not aligned with ITER requirements. Firstly, the implementation is done in Java, while all the rest of the services in ITER require to be implemented in C++, due to efficiency and optimization criteria. Secondly, this implementation is based on a typical architecture (frontend -buffer -backend) but does not have a frontend scheme that allows an acceptable integration of other ITER data types, which would force to maintain several solutions with similar purpose and architecture. Finally, the storage is based on Proto-colBuffers technology that requires prior definition and compilation of the data type, which makes it difficult to store variables continuously, as they may change their structure and data type over time.
Channel Access HDF5 data model EPICS control variables in the Channel Access  [8]  protocol have a composite type format that mainly comprises the following fields: • Status: System status • Severity: Criticality of the alarm if in an alarm state. • EPOCH Timestamp seconds: Section of seconds of the EPOCH timestamp from 1990 to 01-01T00:00:00Z. The data type of the value field can be obtained dynamically through the Channel Access protocol, so it is not necessary to have it defined in some kind of previous configuration, and in fact (as we will see later), it can vary over time. In addition to the variable structure, there is another series of variable metadata that must be taken into account as part of the information to be stored and which are provided dynamically through the Channel Access protocol.
There are two that are of particular importance. The first one, in the case of enumerated type variables, describes the list of values that are encoded. That is, the label represents each of the values of an enumerated variable. The second one describes relevant aspects when displaying the values of the variable, such as the limit values or value unit when representing the variable. In the case of the Channel Access protocol, the data model that has been implemented is presented in Fig. 3. It assigns an HDF5 group to each process variable. In such a group, a "payload" dataset is reserved for values and two additional datasets store display metadata and enumerated type labels. The payload dataset has a compound type with the following fields: "timestamp" (uint64), "status" (int16), "severity" (int16) and "value".
The "value" field will have a datatype that will be aligned with its corresponding PV ("Process Variable") value datatype. This model presents important advantages. The first one is high aggregation rate. Thanks to this model it is possible to aggregate more than 65. 000 variables in a single file. Because the huge number of PVs that are planned for ITER, this property reduces dramatically the complexity of the ITER data archiving system. The second advantage is a high data management granularity. Associating every PV with a payload dataset, every write and read operation will be optimized because it will not be affected by data from other PVs.
pvAccess HDF5 data model EPICS incorporates, from version 7 onwards, the "pvAccess" protocol  [9]. This allows the transmission of control variables (PVs) with a much more complex and highly recursive data structure, which, in the case of ITER, can contain more than 5000 fields in a single control variable. An example of this type of variables can be seen in Fig. 4 , where different nested structures, arrays of structures and common data types, such as: integers, unsigned integers, floats, strings, etc. , can be seen. In order to store this type of nested data, an HDF5 data model has been implemented that conforms to this scheme. Specifically, a recursive model has been created with the following correspondence: • Data structure: An HDF5 group is created. • For each field with a complex data type (non-basic data type): An HDF5 group is created.
This model has several advantages: • There is a 1-to-1 map between the original structure of the variable pvAccess and the HDF5 model. • The fields are accessible, thanks to the HDF5 API, through a simple path that follows the natural structure of the variable. For example based on the example in Fig. 4 , "/main/Sensor  [2] /Integrated/ Quality". • It is easy to access the data in a substructure (a branch of the GroupTree model). • The model is optimized at the data reading level since accessing the values of a certain field only requires reading the data of its dataset, but not the rest of the data.