Python Interface#

Neper’s modules can be run from Python, using the neper Python package, which is provided in the python directory of the distribution. Each module is a function, whose arguments are the module’s command-line options:

import neper

neper.T(n=100, id=1, o="poly")         # neper -T -n 100 -id 1 -o poly
neper.M("poly.tess", elttype="hex")    # neper -M poly.tess -elttype hex
neper.V("poly.tess", datacellcol="id", print="img")
                                       # neper -V poly.tess -datacellcol id -print img

The package runs the neper program (in a separate process), so that it supports all Neper capabilities and requires no compilation, and a Neper error (which ends the neper program) cannot end the Python program, but raises an exception instead. It also reads Neper’s files into NumPy arrays and passes NumPy arrays to Neper, so that tessellations, meshes and simulation results can be processed in Python:

tess = neper.T(n=100, id=1).read("tess")
tess.seed_coo      # seed coordinates, NumPy array of shape (100, 3)

Installation#

The package requires Python 3.8 or higher and NumPy (installed automatically by pip; NumPy is needed only to read files and pass arrays). It is installed using pip, from the python directory of the distribution:

$ pip install ./python

Neper itself must also be installed (see Installation). The neper program is looked for, successively, in environment variable NEPER_EXECUTABLE (which can be the program or its directory), and in the PATH. It can also be set from Python, using neper.set_defaults(executable=...).

Running Modules#

Modules -T, -M, -S and -V are run by functions neper.T, neper.M, neper.S and neper.V, respectively. Their arguments are converted into command-line arguments and passed in order, positional arguments first:

  • A positional argument that is a string (or a path) is passed as is. This is typically the input file of module -M or -V, or the simulation directory of module -S, but it can also be any command-line argument, such as -loop in module -V.

  • A positional argument that is a list (or a tuple) is passed as a comma-separated argument, e.g. ["n10.tess", "n10.msh"] gives n10.tess,n10.msh.

  • A positional argument that is a dictionary is passed as options, in order. This makes it possible to repeat an option, as needed by module -V (see below).

  • A keyword argument is passed as an option: name=value gives -name value. A trailing underscore is removed from the name (so that Python keywords can be used, e.g. in_ for -in). A value of None omits the option, True and False give 1 and 0, a list (or a tuple, or a 1D NumPy array) gives a comma-separated argument, e.g. format=["tess", "vtk"] gives -format tess,vtk, and a 2D array (or a list of lists) is written to a data file and gives file(<file_name>) (see Passing Data). Integers, reals, paths and NumPy scalars are converted to strings.

String arguments are passed to Neper without going through a shell, so that they need no quotes, even when they contain shell metacharacters, e.g. morpho="diameq:lognormal(1,0.35)".

In module -V, options are read in batches that end with -print, so that several images can be generated in one run by passing dictionaries:

neper.V("n10-id1.tess",
        {"datacellcol": "id", "print": "img1"},
        {"datacellcol": "ori", "print": "img2"})

Function neper.run runs Neper with any command-line arguments, provided as a list or as a string (split as by a shell), as in neper.run("-T -n 100 -id 1").

Run Parameters#

The following keyword arguments are not passed to Neper, but set how it is run:

  • cwd: working directory, i.e. where the input files are read and the output files are written (default: the current directory);

  • threads: number of OpenMP threads (default: OMP_NUM_THREADS, or all threads, see Introduction);

  • rcfile: initialization file (default: $HOME/.neperrc); "none" ignores the initialization file, which makes results independent of the user’s settings;

  • verbose: print Neper’s messages while it runs (default: False);

  • check: raise an exception when Neper fails (default: True);

  • env: environment variables to set (or, for a value of None, to unset) for Neper, as a dictionary;

  • executable: neper program.

Their default values can be changed using neper.set_defaults, as in neper.set_defaults(threads=8, rcfile="none", verbose=True).

Results#

A function returns a result, whose main attributes are:

  • files: the files written by Neper, as a list of absolute paths (pathlib.Path), in the order they were written. Method file returns the (first) written file of a given extension, or the first written file (the main output of the module), as in neper.T(n=100).file("tess");

  • output: Neper’s messages (a string);

  • returncode: the exit status of neper (0 on success);

  • cmd: the command line (a list of strings);

  • elapsed: the elapsed time (in seconds).

The output files can then be used as input of another module, or read in Python, using method read, which reads the (first) written file of a given extension, or the first written file (see Reading Files), as in neper.T(n=100).read("tess"). Method stat reads the statistics of an entity as a dictionary indexed by the keys of the corresponding -stat* option, as in:

res = neper.T(n=100, statcell="vol,sphericity")
res.stat("cell")["vol"]       # NumPy array

In Jupyter, a result that includes an image (e.g., of module -V) is displayed as the image.

Reading Files#

Neper’s files are read into NumPy arrays by function neper.read, which calls, depending on the file extension:

  • neper.read_tess for a tessellation file (.tess, see Tessellation File (.tess)), which returns a neper.Tess;

  • neper.read_tesr for a raster tessellation file (.tesr, see Raster Tessellation File (.tesr)), which returns a neper.Tesr;

  • neper.read_mesh for a mesh file (.msh, ASCII or binary, see Mesh File (.msh)), which returns a neper.Mesh;

  • neper.read_sim for a simulation directory (see Simulation Directory (.sim)), which returns a neper.Sim (see Simulation Directories);

  • neper.read_odf for an orientation distribution function file (.odf, see Orientation Distribution Function File (.odf)), which returns a neper.Odf (with method mesh to read the corresponding orientation-space mesh);

  • neper.read_stat for a statistics file (.st*) or another table of numbers (e.g., an orientation file, .ori), which returns an array with one row per entity and one column per value (or a list of arrays if the numbers of values differ, as for faces). If a list of keys is provided, as in neper.read_stat("n100-id1.stcell", "vol,sphericity"), a dictionary (key → values) is returned instead.

Entities are numbered from 1, as in Neper: row i of an array describes entity i + 1, and references to entities are identifiers. For example, the coordinates of the vertices of edge 1 are tess.ver_coo[tess.edge_vers[0] - 1]. The attribute names follow Neper’s structures and keys (seed, cell, ver, edge, face, poly, node, elt, elset, etc.), and print (x) lists the attributes of an object x.

A neper.Tess contains:

  • the cells: cell_qty, seed_coo (shape (n, 3)), seed_weight, cell_ori (shape (n, k)), cell_orides (the orientation descriptor, such as rodrigues:passive), cell_crysym, and, if any, cell_id, cell_lam, cell_mode, cell_group, cell_orispread and cell_weight;

  • the vertices, edges, faces and polyhedra: ver_coo (shape (n, 3)), edge_vers (shape (n, 2)), face_vers, face_edges, face_eq (shape (n, 4), d, a, b and c, such that \(a \, x + b \, y + c \, z = d\)) and poly_faces, where face_vers, face_edges and poly_faces are lists of arrays (with signed edges and faces, as in the file) and ver_state, edge_state, face_state, face_pt and face_ptcoo are also available;

  • the domain: dom_type, dom_ver_*, dom_edge_* and dom_face_* (such as dom_face_label and dom_face_tessfaces);

  • the periodicity (if any): periodic, periodic_dist, per_seed, per_ver, per_edge and per_face, and the scales (if any): scale_qty and cell_scaleid.

A neper.Tesr contains size, vsize and origin, the cells of the voxels, vox_cell, an integer array indexed as [x, y, z] (or [x, y] in 2D), the voxel orientations, vox_ori (indexed as [x, y, z, k]), vox_orides and vox_oridef (if any), and the cell data (cell_qty, seed_coo, cell_ori, etc.).

A neper.Mesh contains the node coordinates, node_coo (shape (n, 3)), and the elements, as blocks of elements of a same type, blocks. Method elts returns the block of a given dimension (default: the mesh dimension), whose attributes are type (such as tet or hex), order, elt_id, elt_nodes (shape (n, k)), elt_elset and elt_part. The mesh also contains (if any) nsets and fasets (dictionaries indexed by label), node_part, periodicity, elset_ori, elset_orides, elset_crysym, elt_ori, elt_orides, elset_group and physical_names.

For example:

import numpy as np

tess = neper.T(n=100, id=1).read()
# vertices of polyhedron 1
faces = np.abs(tess.poly_faces[0])
vers = np.unique(np.concatenate([tess.face_vers[f - 1] for f in faces]))

mesh = neper.M("n100-id1.tess", elttype="tet", order=1).read()
elts = mesh.elts()
centroids = mesh.node_coo[elts.elt_nodes - 1].mean(axis=1)

vol, sph = neper.read_stat(neper.T(n=100, statcell="vol,sphericity").file("stcell")).T

Simulation Directories#

A simulation directory (see Simulation Directory (.sim)) is read by neper.read_sim (or neper.read), which returns a neper.Sim, and its results are read on demand. A neper.Sim provides:

  • the input files, inputs (a dictionary indexed by extension, such as msh), which are read by methods tess, tesr and mesh (once);

  • the result entities, entities (such as nodes, elts, elsets and mesh), and the steps, steps;

  • the orientation descriptor of the orientation results, orides (that of the input mesh or tessellation);

  • method results, which returns the results of an entity, and methods result_type and result_steps, which return the type (from the definition file or for a result of known type, see Simulation Results) and the steps of a result;

  • method read, which reads a result at a step, as an array of shape (n, k) ((n,) for a single component), or at all steps, as an array of shape (steps, n, k) (or (steps, n)), and method read_all, which reads all results, as a dictionary (entity → result → array);

  • method write, which writes a result (at a step or all steps, as for read), with its definition file, so that it can be used as any other result by modules -S and -V (a result used in an expression of module -S must be written for all steps), and method remove, which removes a result.

Results can then be computed by module -S (neper.S) or in Python, and plotted or visualized by module -V. For example:

import numpy as np
import matplotlib.pyplot as plt

neper.S("n100.sim", resnode="disp", reselset="vol")
sim = neper.read("n100.sim")

disp = sim.read("nodes", "disp")             # (steps, nodes, 3)
top = sim.mesh().nsets["z1"] - 1
plt.plot(sim.steps, disp[:, top, 2].mean(axis=1))

vol = sim.read("elsets", "vol")              # (steps, elsets)
sim.write("elsets", "volchange", vol / vol[0] - 1)
neper.V("n100.sim", step=sim.steps[-1], dataelsetcol="real:volchange", print="img")

When the simulation directory is modified by module -S, method refresh updates the entities and steps.

Orientations#

Orientations are converted between descriptors (see Rotations and Orientations) by neper.ori_convert, following Neper’s conventions:

tess = neper.read("n100-id1.tess")
euler = neper.ori_convert(tess.cell_ori, tess.cell_orides, "euler-bunge")

The descriptors are rodrigues, euler-bunge, euler-kocks, euler-roe, rotmat (9 components, row by row), axis-angle and quaternion, optionally followed by the convention (:passive, the default, or :active). Angles are in degrees, and the values are those that Neper writes (e.g., Euler angles in \([0,\,360) \times [0,\,180] \times [0,\,360)\)). The arrays can have any shape, with the parameters along the last axis. neper.ori_angle returns the rotation angle between orientations (not accounting for crystal symmetry), as in neper.ori_angle(ori, ori[0], sim.orides) for the rotations of the grains of a simulation from step 0.

Orilib#

The functions of Orilib, Neper’s library of orientation functions, are available in Python in module neper.orilib, under their Orilib names (see Orilib Commands). They apply to NumPy arrays:

from neper import orilib

orilib.ol_e_q([0, 30, 0])                     # quaternion, shape (4,)
q = orilib.ol_e_q(euler)                      # euler: shape (n, 3) -> (n, 4)
orilib.ol_q_q_disori(q, q[0], "cubic")        # disorientation angles, shape (n,)
orilib.ol_q_crysym(q, "cubic", 7)             # equivalent orientations, shape (n, 4)

Each argument is an array whose last axis holds the values of one item, e.g. the 4 components of a quaternion (or whose two last axes hold a rotation matrix), and whose leading axes index the items. The leading axes of the arguments are broadcast (as in NumPy), so that a single item can be combined with several items, as q[0] above. An integer or string argument (e.g. a crystal symmetry) applies to all items, or is given per item (as an array). The results have the leading axes, followed by the axes of an item. A function that returns several values returns a tuple, and a function that returns a list of values whose length differs between the items (e.g. ol_polef_polecrysym) returns a list of the item results.

The arguments and results of a function are given by orilib.signature, as in orilib.signature("ol_q_q_disori"), which gives ol_q_q_disori (q1: q, q2: q, crysym: str) -> theta: theta, and by help (orilib.ol_q_q_disori), which also prints the description of the corresponding C function. The types are those of Orilib: e (Euler angles, in degrees), g (rotation matrix, shape (3, 3)), q (quaternion), R (Rodrigues vector), rtheta (rotation axis / angle pair), m (Miller indices), theta (angle), vect (vector), pole, etc. orilib.commands lists the functions.

The functions and their arguments are those of the orilib program (see orilib signatures). They run the program (once per call, for all items, about 0.5 s for 100,000 items), which is looked for, successively, in environment variable ORILIB_EXECUTABLE, next to the neper program, and in the PATH. An error of Orilib (e.g. an unknown crystal symmetry) raises a neper.orilib.OrilibError exception. The functions that do not apply to individual items (ol_g_m, ol_q_m, ol_set_filter and ol_set_filter_iter) are not available, but any command can be run with orilib.run, which takes and returns text.

Passing Data#

Data are passed to Neper as data files (see Data File), and NumPy arrays can be written to data files using function neper.datafile, which returns an object that gives file(<file_name>) when passed to Neper, as an option value or in a string. The data file is written to a temporary directory (deleted when Python exits), or to a given path (argument path). A 2D array (or a list of lists) passed as an option value is written to a data file automatically. For example:

import numpy as np

ori = np.random.uniform(-0.3, 0.3, (100, 3))          # Rodrigues vectors
neper.T(n=100, ori=ori)                               # -ori file(<file_name>)

seeds = np.random.random((100, 3))
neper.T(n=100, morphooptiini=f"coo:{neper.datafile(seeds)}")

tess = neper.read("n100-id1.tess")
neper.V("n100-id1.tess", datacellcol=f"real:{neper.datafile(tess.seed_coo[:, 2])}",
        print="img")

Errors#

When Neper fails (e.g., because of an invalid option or a missing file), a neper.NeperError exception is raised. Its message is Neper’s error message, and its attributes include returncode and output (Neper’s messages):

try:
    neper.T(n=100, morpho="foo")
except neper.NeperError as e:
    print(e.message)    # Unknown argument `foo'.

Using check=False, no exception is raised and the result can be tested instead (attribute ok).

Interrupting Python (Ctrl+C) interrupts Neper, which then behaves as when it is interrupted from a terminal: during the morphology optimization of module -T, the optimization stops and Neper continues and completes (and the function returns normally), while at other stages Neper stops (and a KeyboardInterrupt exception is raised). A second interruption stops Neper in all cases.

Example#

The following script generates a polycrystal with a grain size distribution, meshes it, and prints an image of the mesh colored by grain orientation (see also Using Neper from Python):

import neper

neper.set_defaults(rcfile="none")

tess = neper.T(n=100, id=1, morpho="diameq:lognormal(1,0.35),1-sphericity:lognormal(0.145,0.03)",
               reg=True).file("tess")
mesh = neper.M(tess, rcl=0.7).file("msh")
neper.V([tess, mesh], dataelsetcol="ori", print="img")

Since Neper is run as a separate process, several Neper runs can also be carried out in parallel, e.g. using concurrent.futures.ThreadPoolExecutor (with a reduced number of threads each, and different output files).