A singular Riemannian geometry approach to Deep Neural Networks.
How to use the example code

To avoid recompiling each time we run a different numerical experiment, the test program reads the information about the dataset and the algorithm to use in the "settings" text file you find in the "init" folder. These data are saved by the program in the settings_info struct.

The settings file

The "settings" file is a simple text file whose entries are separated by a tab ('\t') character. The test code allows to perform the following tasks:

  • To predict the outputs of a dataset.
  • To run the SiMEC-1D algorithm, in order to build the equivalence class of a given point.
  • To run the SiMExp-1D algorithm, in order to build part of a foliation of the input manifold starting from a given point.

The order of the arguments follows that of the fields in the settings_info struct, namely:

  • The name of the file containing the dataset (if none, write '#'). This argument is needed only to run the SiMEC-1D and SiMExp-1D algorithms.
  • The size of the input variables.
  • The size of the output variables.
  • Whether data must be normalized or not ('y'/'n').
  • The name of the file containing the structure of the neural network and its weights/biases.
  • The name of the file in which the SiMEC algorithms shall write their outputs.
  • The number of iterations.
  • The delta of the SiMEC-1D algorithm.
  • Whether to invert the direction of SiMEC-1D or not ('y'/'n'). Note that there is no privileged direction. The direction in which the curve is built is selected during the first iteration is the direction of the lowest eigenvector of the (degenerate) metric induced on the input manifold , which depends on the particular algorithm employed to find the eigenvectors. This direction correspond to 'y', the opposite one to 'n'. In practice, for 1D equivalence sets, one choice makes the algorithm build a curve from left to right, the other option in the opposite direction. Our advice is to run the algorithm two times using both the options, in order to get a curve in both directions.
  • normalize_starting: If SiMEC-1D or SiMExp-1D are selected, this parameter specifies whethere the dataset must be normalized or not (true/false). If true, the maximum and the minimum of the dataset the starting point comes from must be provided in a max_mix_struct.
  • The file containing the point for which we build the equivalence class.
  • The algorithm to run: 'SiMEC-1D' / 'SiMExp-1D' / 'Predict'.
  • The epsilon of SiMExp-1D.
  • delta_simexp: Delta of the SiMExp-1D algorithm, namely the maximum distance from the starting point.

For example the following settings

<# 2 1 n weights/weights_binary.csv output/test/simexp_class_out.csv 10000 1.e-3 n init/starting_point n SiMExp-1D 0.1 5.e-6>

make the test program run the SiMExp-1D, for which no dataset is needed, starting from the point in "init/starting_point", initializing the neural network as specified in "weights/weights_binary.csv" and saving the output points "in output/test/simexp_class_out.csv" with the following parameters:

  • The size of the input vector is 2.
  • The size of the output space is 1.
  • Data must not be normalized.
  • The algorithm is to be executed for 10000 iterations.
  • The Delta of the SiMEC-1D algorithm is 1.e-3
  • The direction in which the curve null is constructed is the opposite with respect to the direction of the lowest eigenvector computed in the first iteration.
  • The starting point is not to normalize.
  • The epsilon of SiMExp-1D is 0.1.
  • The delta of the SiMExp-1D algorithm is 5.e-6.

Scripts to plot the results of SiMEC and SiMExp

In the output folder, there are the Python scripts employed to analyze the numerical experiments carried out in the paper "A singular Riemannian geometry approach to Deep Neural Networks II. Reconstruction of 1-D equivalence classes.".

  • analysis_exp_1 : Script to generate the plot for the first numerical experiment - "Learning compact equivalence classes".
  • analysis_exp_2 : Script to generate the plot for the second numerical experiment - "Learning non compact equivalence classes".
  • analysis_thermodynamics: Script to generate the plot for the third numerical experiment - "A thermodynamics problem".
  • analysis_exp_annulus : Script to generate the plot for the fourth numerical experiment - "Learning preimages of compact equivalence classes".
  • analysis_thermo_between_iso.py : Script to generate the plot for the fifth numerical experiment - "Thermodynamics: learning a family of isothermal curves".
  • analysis_binary.py : Script to generate the plot for the sixth numerical experiment - "A classification problem".