ptypy.utils package¶
Submodules¶
ptypy.utils.array_utils module¶
utility functions to manipulate/reshape numpy arrays.
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- ptypy.utils.array_utils.c_zoom(c, *arg, **kwargs)¶
Deprecated, kept for backward compatibility only.
Wrapper scipy.ndimage.zoom function and shares the same input arguments.
It performs a complex overlaod if the input is complex.
- Parameters
c (numpy.ndarray) – Array to shiftzoom. Can be float or complex
- Returns
Zoomed array
- Return type
numpy.ndarray
- ptypy.utils.array_utils.crop_pad(A, hplane_list, axes=None, cen=None, fillpar=0.0, filltype='scalar')¶
Crops or pads a volume array A with a number of hyperplanes according to parameters in hplanes Wrapper for crop_pad_axis.
- Parameters
A (array-like) – input array of any shape
hplane_list (array-like, list) – list of scalars or tupels counting the number of hyperplanes to crop / pad see crop_pad_axis() for detail. If N=len(hplane_list) has less entries than dimensions of A, the last N axes are used
axes (tuple, list) – tuple / list of axes to be used for cropping / padding, has to be same length as hplanes
cen – center of array, padding/cropping occurs at cen + A.shape / 2
- Returns
Cropped or padded array.
- Return type
array-like
See also
Examples
>>> import numpy as np >>> from ptypy.utils import crop_pad >>> V=np.random.rand(3,5,5)
Pads 4 planes of zeros on the last axis (2 on low side and 2 on high side) and pads 3 planes of zeros on the second last axis (2 on low side and 1 on high side):
>>> B=crop_pad(V,[3,4])
Also equivalent:
>>> B=crop_pad(V,[(2,1),(2,2)]) >>> B=crop_pad(V,[(2,1),(2,2)], axes=[-2,-1],fillpar=0.0,filltype='scalar')
None-peripheral cropping /padding:
>>> from ptypy.utils import grids >>> fftshift_grid = grids((6,4),center = 'fft') >>> cropped_grid=crop_pad(V,[-2,4],cen='fft')
Note that cropping/ padding now occurs at the start and end of fourier coordinates useful for cropping /padding high frequencies in fourier space.
- ptypy.utils.array_utils.crop_pad_axis(A, hplanes, axis=- 1, roll=0, fillpar=0.0, filltype='scalar')¶
Crops or pads a volume array A at beginning and end of axis axis with a number of hyperplanes specified by hplanes
- Parameters
A (array-like) – Input array, should be at least one-dimensional
hplanes (int or tuple of int) – Number of hyperplanes to add or remove in total. If tuple of 2 ints, this value is interpreted as (begin, end). See the
Note
for more information.axis (int, optional) – axis to be used for cropping / padding
roll (int,) – Roll array backwards by this number prior to padding / cropping. The roll is reversed afterwards
fillpar – See
pad_lr
for explqanationfilltype – See
pad_lr
for explqanation
- Returns
Cropped / padded array
- Return type
array-like
Note
- if hplanes is scalar and negativ :
crops symmetrically, low-index end of axis is preferred if hplane is odd,
- if hplanes is scalar and positiv :
pads symmetrically with a fill specified with ‘fillpar’ and ‘filltype’ look at function pad_lr() for detail.
- if hplanes is tupel :
function pads /crops at (begin, end) according to the tupel.
Examples
>>> import numpy as np >>> from ptypy.utils import crop_pad_axis >>> A=np.ones((8,9))
Add a total of 2 rows, one at top, one at bottom.
>>> B=crop_pad_axis(A,2,0)
That is the same as
>>> B = crop_pad_axis(A,(1,1),0))
Crop 3 columns on the left side and pad 2 columns on the right:
>>> B = crop_pad_axis(A,(-3,2),1)
Crop one plane on low-side and high-side (total of 2) of Volume V:
>>> V=np.random.rand(3,5,5) >>> B=crop_pad_axis(V,-2,0)
Mirror volume 3 planes on low side of row axis, crop 2 planes on high side:
>>> B=crop_pad_axis(V,(3,-2),1,filltype='mirror')
- ptypy.utils.array_utils.crop_pad_symmetric_2d(A, newshape, center=None)¶
Crops or pads Array A symmetrically along the last two axes (-2,-1) around center center to a new shape newshape.
- Parameters
A (array-like) – Input array, must be at least two-dimensional.
newshape (tuple, array_like) – New shape (for the last two axes). Must have at least 2 entries.
center (tuple, array_like, optional) – This coordinate tuple marks the center for cropping / padding If None,
np.array(A.shape[-2:]) // 2
will be the center
- Returns
out (ndarray) – Cropped or padded array with shape
A.shape[:-2]+newshape[-2:]
center (tuple) – Center in returned array, should be in the actual center of array.
See also
- ptypy.utils.array_utils.ellipsis(grids, dims=None, ew=2)¶
- ptypy.utils.array_utils.grids(sh, psize=None, center='geometric', FFTlike=True)¶
q0,q1,... = grids(sh)
returns centered coordinates for a N-dimensional array of shape sh (pixel units)q0,q1,... = grids(sh,psize)
gives the coordinates scaled according to the given pixel size psize.q0,q1,... = grids(sh,center='fftshift')
gives the coordinates shifted according to fftshift convention for the originq0,q1,... = grids(sh,psize,center=(c0,c1,c2,...))
gives the coordinates according scaled with psize having the origin at (c0,c1,..)- Parameters
sh (tuple of int) – The shape of the N-dimensional array
psize (float or tuple of float) – Pixel size in each dimensions
center (tupel of int) – Tuple of pixel, or use
center='fftshift'
for fftshift-like grid andcenter='geometric'
for the matrix center as grid originFFTlike (bool) – If False, grids ar not bound by the interval [-sh//2:sh//2[
- Returns
The coordinate grids
- Return type
ndarray
- ptypy.utils.array_utils.mirror(A, axis=- 1)¶
Mirrors array A along one axis axis
- Parameters
A (array-like) – Input array
axis (int, optional) – Axis along which the array will be mirrored
- Returns
A view to the mirrored array.
- Return type
array-like
- ptypy.utils.array_utils.pad_lr(A, axis, l, r, fillpar=0.0, filltype='scalar')¶
Pads ndarray A orthogonal to axis with l layers (pixels,lines,planes,…) on low side an r layers on high side.
- Parameters
A (array-like) – Input array
axis (int) – Pads orthogonal to axis on with l layers (pixels,lines, planes,…) on low side an r layers on high side.
l (int) – Pads orthogonal to axis on with l layers (pixels,lines, planes,…) on low side an r layers on high side.
r (int) – Pads orthogonal to axis on with l layers (pixels,lines, planes,…) on low side an r layers on high side.
fillpar (scalar) – Scalar fill parameter for
filltype=scalar
fill.filltype (str) –
‘scalar’, uniformly pad with fillpar
’mirror’, mirror A
’periodic’, periodic fill
’custom’, pad according arrays found in fillpar
- Returns
Padded array
- Return type
ndarray
See also
- ptypy.utils.array_utils.rebin(a, *args, **kwargs)¶
Rebin ndarray data into a smaller ndarray of the same rank whose dimensions are factors of the original dimensions.
Note
eg. An array with 6 columns and 4 rows can be reduced to have 6,3,2 or 1 columns and 4,2 or 1 rows.
- Parameters
a (ndarray) – Input array.
axis (int, Default=-1, optional) – The laplacian is computed along the provided axis or list of axes, or all axes if None
- Returns
out – Rebinned array.
- Return type
ndarray
Examples
>>> import ptypy >>> import numpy as np >>> a=np.random.rand(6,4) >>> b=ptypy.utils.rebin(a,3,2) a.reshape(args[0],factor[0],args[1],factor[1],).sum(1).sum(2)*( 1./factor[0]/factor[1]) >>> a2=np.random.rand(6) >>> b2=ptypy.utils.rebin(a2,2) a.reshape(args[0],factor[0],).sum(1)*( 1./factor[0])
- ptypy.utils.array_utils.rebin_2d(A, rebin=1)¶
Rebins array A symmetrically along the last 2 axes with factor rebin.
- Parameters
A (array-like) – input array, must be at least two-dimensional.
rebin (int) – rebin factor,
rebin=2
means that a square of 4 pixels will be binned into one.
- Returns
out – rebinned array. Note that data type casting follows numpy rules, so that boolean arrays are converted to int.
- Return type
ndarray
See also
- ptypy.utils.array_utils.rectangle(grids, dims=None, ew=2)¶
- ptypy.utils.array_utils.shift_zoom(c, zoom, cen_old, cen_new, **kwargs)¶
Move array from center cen_old to cen_new and perform a zoom zoom.
This function wraps scipy.ndimage.affine_transform and uses the same keyword arguments.
Addiionally, it allows for complex input and out by complex overloading, see
complex_overload
.- Parameters
c (numpy.ndarray) – Array to shiftzoom. Can be float or complex
zoom (float) – Zoom factor
cen_old (array_like) – Center in input array c
cen_new (array_like) – Desired new center position in shiftzoomed array
- Returns
Shifted and zoomed array
- Return type
numpy.ndarray
- ptypy.utils.array_utils.switch_orientation(A, orientation, center=None)¶
Switches orientation of Array A along the last two axes (-2,-1)
- Parameters
A (array-like) – input array, must be at least twodimensional
orientation (tuple or int) –
3-tuple of booleans (transpose,flipud,fliplr). If integer, value is converted according to binary representation:
0: [False, False, False] 1: [False, False, True] 2: [False, True, False] 3: [False, True, True] 4: [True, False, False] 5: [True, False, True] 6: [True, True, False] 7: [True, True, True]
center (tuple, optional) – move this coordinate alomg with the transformations
- Returns
out (ndarray) – A view of A where either rows, columns and axis may be reversed
center (tuple) – new center
- ptypy.utils.array_utils.zoom(c, *arg, **kwargs)¶
Deprecated, kept for backward compatibility only.
Wrapper scipy.ndimage.zoom function and shares the same input arguments.
It performs a complex overlaod if the input is complex.
- Parameters
c (numpy.ndarray) – Array to shiftzoom. Can be float or complex
- Returns
Zoomed array
- Return type
numpy.ndarray
ptypy.utils.math_utils module¶
Numerical util functions.
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- ptypy.utils.math_utils.abs2(A)¶
Squared absolute value for an array A.
- ptypy.utils.math_utils.c_gf(input, sigma, order=0, output=None, mode='reflect', cval=0.0, truncate=4.0)¶
complex input
Multidimensional Gaussian filter.
- Parameters
input (array_like) – The input array.
sigma (scalar or sequence of scalars) – Standard deviation for Gaussian kernel. The standard deviations of the Gaussian filter are given for each axis as a sequence, or as a single number, in which case it is equal for all axes.
order (int or sequence of ints, optional) – The order of the filter along each axis is given as a sequence of integers, or as a single number. An order of 0 corresponds to convolution with a Gaussian kernel. A positive order corresponds to convolution with that derivative of a Gaussian.
output (array or dtype, optional) – The array in which to place the output, or the dtype of the returned array. By default an array of the same dtype as input will be created.
mode (str or sequence, optional) –
The mode parameter determines how the input array is extended when the filter overlaps a border. By passing a sequence of modes with length equal to the number of dimensions of the input array, different modes can be specified along each axis. Default value is ‘reflect’. The valid values and their behavior is as follows:
- ’reflect’ (d c b a | a b c d | d c b a)
The input is extended by reflecting about the edge of the last pixel. This mode is also sometimes referred to as half-sample symmetric.
- ’constant’ (k k k k | a b c d | k k k k)
The input is extended by filling all values beyond the edge with the same constant value, defined by the cval parameter.
- ’nearest’ (a a a a | a b c d | d d d d)
The input is extended by replicating the last pixel.
- ’mirror’ (d c b | a b c d | c b a)
The input is extended by reflecting about the center of the last pixel. This mode is also sometimes referred to as whole-sample symmetric.
- ’wrap’ (a b c d | a b c d | a b c d)
The input is extended by wrapping around to the opposite edge.
For consistency with the interpolation functions, the following mode names can also be used:
- ’grid-constant’
This is a synonym for ‘constant’.
- ’grid-mirror’
This is a synonym for ‘reflect’.
- ’grid-wrap’
This is a synonym for ‘wrap’.
cval (scalar, optional) – Value to fill past edges of input if mode is ‘constant’. Default is 0.0.
truncate (float) – Truncate the filter at this many standard deviations. Default is 4.0.
- Returns
gaussian_filter – Returned array of same shape as input.
- Return type
ndarray
Notes
The multidimensional filter is implemented as a sequence of 1-D convolution filters. The intermediate arrays are stored in the same data type as the output. Therefore, for output types with a limited precision, the results may be imprecise because intermediate results may be stored with insufficient precision.
Examples
>>> from scipy.ndimage import gaussian_filter >>> a = np.arange(50, step=2).reshape((5,5)) >>> a array([[ 0, 2, 4, 6, 8], [10, 12, 14, 16, 18], [20, 22, 24, 26, 28], [30, 32, 34, 36, 38], [40, 42, 44, 46, 48]]) >>> gaussian_filter(a, sigma=1) array([[ 4, 6, 8, 9, 11], [10, 12, 14, 15, 17], [20, 22, 24, 25, 27], [29, 31, 33, 34, 36], [35, 37, 39, 40, 42]])
>>> from scipy import misc >>> import matplotlib.pyplot as plt >>> fig = plt.figure() >>> plt.gray() # show the filtered result in grayscale >>> ax1 = fig.add_subplot(121) # left side >>> ax2 = fig.add_subplot(122) # right side >>> ascent = misc.ascent() >>> result = gaussian_filter(ascent, sigma=5) >>> ax1.imshow(ascent) >>> ax2.imshow(result) >>> plt.show()
- ptypy.utils.math_utils.cabs2(A)¶
Squared absolute value for an array A. If A is complex, the returned value is complex as well, with the imaginary part of zero.
- ptypy.utils.math_utils.delxb(a, axis=- 1)¶
Backward first order derivative for finite difference calculation.
Note
The first element along the derivative direction is set to 0.
Pixel units are used (\(\Delta x = \Delta h = 1\)).
- Parameters
a (ndarray) – Input array.
axis (int, Default=-1, optional) – Which direction used for the derivative.
- Returns
out – Derived array.
- Return type
ndarray
- ptypy.utils.math_utils.delxc(a, axis=- 1)¶
Central first order derivative for finite difference calculation.
Note
Forward and backward derivatives are used for first and last elements along the derivative direction.
Pixel units are used (\(\Delta x = \Delta h = 1\)).
- Parameters
a (nd-numpy-array) – Input array.
axis (int, Default=-1, optional) – Which direction used for the derivative.
- Returns
out – Derived array.
- Return type
nd-numpy-array
- ptypy.utils.math_utils.delxf(a, axis=- 1, out=None)¶
Forward first order derivative for finite difference calculation.
Note
The last element along the derivative direction is set to 0.
Pixel units are used (\(\Delta x = \Delta h = 1\)).
- Parameters
a (ndarray) – Input array.
axis (int, Default=-1, optional) – Which direction used for the derivative.
out (ndarray, Default=None, optional) – Array in wich the resault is written (same size as
a
).
- Returns
out – Derived array.
- Return type
ndarray
- ptypy.utils.math_utils.gauss_fwhm(x, fwhm=1.0, off=0.0)¶
Evaluates gaussian with full width half maximum
- Parameters
x (ndarray) – input array
fwhm (float,optional) – Full width at half maximum
off (float, optional) – Offset / shift
See also
- ptypy.utils.math_utils.gaussian(x, std=1.0, off=0.0)¶
Evaluates gaussian standard normal
\[g(x)=\frac{1}{\mathrm{std}\sqrt{2\pi}}\,\exp \left(-\frac{(x-\mathrm{off})^2}{2 \mathrm{std}^2 }\right)\]- Parameters
x (ndarray) – input array
std (float,optional) – Standard deviation
off (float, optional) – Offset / shift
See also
- ptypy.utils.math_utils.gaussian2D(size, std_x=1.0, std_y=1.0, off_x=0.0, off_y=0.0)¶
Evaluates normalized 2D gaussian on array of dimension size. Origin of coordinate system is in the center of the array.
- Parameters
size (int) – length of requested array
std_x (float,optional) – Standard deviation in x direction
std_y (float,optional) – Standard deviation in y direction
off_x (float, optional) – Offset / shift in x direction
off_y (float, optional) – Offset / shift in y direction
- ptypy.utils.math_utils.gf(c, *arg, **kwargs)¶
Wrapper for scipy.ndimage.gaussian_filter, that determines whether original or the complex function shall be used.
See also
- ptypy.utils.math_utils.gf_2d(c, sigma, **kwargs)¶
Gaussian filter along the last 2 axes
- ptypy.utils.math_utils.norm(A)¶
Norm.
- ptypy.utils.math_utils.norm2(A)¶
Squared norm
- ptypy.utils.math_utils.ortho(modes)¶
Orthogonalize the given list of modes or ndarray along first axis. specify procedure
- Parameters
modes (array-like or list) – List equally shaped arrays or array of higher dimension
- Returns
amp (vector) – relative power of each mode
nplist (list) – List of modes, sorted in descending order
- ptypy.utils.math_utils.rl_deconvolution(data, mtf, numiter)¶
Richardson Lucy deconvolution on a 2D numpy array.
- Parameters
data (2darray) – Diffraction data (measured intensity).
mtf (2darray) – Modulation transfer function (Fourier transform of detector PSF).
numiter (int) – Number of iterations.
- Returns
out (2darray) – Approximated intensity after numiter iterations.
Note
Assumes that mtf is symmetric and that data is real and positive.
mtf is non fft shifted that means that the dc component is on mtf[0,0].
Todo
non symmetric mtf (mtf.conj()[-q] somewhere)
optimisation (FFTW? scipy fft? error metric cancel iter?)
Original code provided by M. Stockmar
- ptypy.utils.math_utils.smooth_step(x, mfs)¶
Smoothed step function with fwhm mfs Evaluates the error function scipy.special.erf.
ptypy.utils.misc module¶
Miscellaneous utility functions with little dependencies from other modules
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- class ptypy.utils.misc.Table(dct, name='pods')¶
Bases:
object
Basic table implemented using numpy.recarray Ideally subclassed to be used with faster or more flexible databases.
- add_records(records)¶
Add records at the end of the table.
- insert_records(records, record_ids)¶
Insert records and overwrite existing content.
- modify_add(record_ids, **kwargs)¶
Take selected record ids and overwrite fields with values **kwargs.
- new_fields(**kwargs)¶
Add fields (columns) to the table. This is probably slow.
- new_table(records=0)¶
- pull_records(record_ids=None)¶
- select_func(func, fields=None)¶
Find all records where search function func evaluates True. Arguments to the function are selected by fields. The search function will always receive the record_id as first argument.
- select_match(field, match)¶
Find all records whose values are in the range [low,`high`] for the field entry field
- select_range(field, low, high)¶
Find all records whose values are in the range [low,`high`] for the field entry field. Should be a numerical value.
- ptypy.utils.misc.all_subclasses(cls, names=False)¶
Helper function for finding all subclasses of a base class. If names is True, returns the names of the classes rather than their object handles.
- ptypy.utils.misc.clean_path(filename)¶
Makes path absolute and create all sub directories if needed.
- Parameters
filename (str) – A filename.
- ptypy.utils.misc.complex_overload(func)¶
Overloads function specified only for floats in the following manner
\[\mathrm{complex\_overload}\{f\}(c) = f(\Re(c)) + \mathrm{i} f(\Im(c))\]
- ptypy.utils.misc.electron_wavelength(electron_energy)¶
Calculate electron wavelength based on energy in keV:
\[\lambda = hc / \sqrt{E * (2moc^2 + E)}\]- Parameters
electron_energy (float) – in units of keV
- Returns
wavelength – in units of meter
- Return type
float
- ptypy.utils.misc.expect2(a)¶
Generates 1d numpy array with 2 entries generated from multiple inputs (tuples, arrays, scalars). Main puprose of this function is to circumvent debugging of input as very often a 2-vector is requred in scripts
Examples
>>> from ptypy.utils import expect2 >>> expect2( 3.0 ) array([ 3., 3.]) >>> expect2( (3.0,4.0) ) array([ 3., 4.]) >>> expect2( (1.0, 3.0,4.0) ) array([ 1., 3.])
- ptypy.utils.misc.expect3(a)¶
Generates 1d numpy array with 3 entries generated from multiple inputs (tuples, arrays, scalars). Main puprose of this function is to circumvent debugging of input.
Examples
>>> from ptypy.utils import expect3 >>> expect3( 3.0 ) array([ 3., 3., 3.]) >>> expect3( (3.0,4.0) ) array([ 3., 4., 4.]) >>> expect3( (1.0, 3.0,4.0) ) array([ 1., 3., 4.])
- ptypy.utils.misc.expectN(a, N)¶
- ptypy.utils.misc.isstr(s)¶
This function should be used for checking if an object is of string type
- ptypy.utils.misc.keV2m(keV)¶
Convert photon energy in keV to wavelength (in vacuum) in meters.
- ptypy.utils.misc.keV2nm(keV)¶
Convert photon energy in keV to wavelength (in vacuum) in nanometers.
- ptypy.utils.misc.m2keV(m)¶
Convert wavelength in meters to photon energy in keV.
- ptypy.utils.misc.nm2keV(nm)¶
Convert wavelength in nanometers to photon energy in keV.
- ptypy.utils.misc.str2int(A)¶
Transforms numpy array A of strings to
np.uint8
and backExamples
>>> from ptypy.utils import str2int >>> A=np.array('hallo') >>> A array('hallo', dtype='|S5') >>> str2int(A) array([104, 97, 108, 108, 111], dtype=uint8) >>> str2int(str2int(A)) array('hallo', dtype='|S5')
- ptypy.utils.misc.str2range(s)¶
Generates an index list from string input s
Examples
>>> # Same as range(1,4,2) >>> str2range('1:4:2') >>> # Same as range(1,2) >>> str2range('1')
- ptypy.utils.misc.unique_path(filename)¶
Makes path absolute and unique
- Parameters
filename (str) – A filename.
ptypy.utils.parallel module¶
Utility functions and classes to support MPI computing.
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- class ptypy.utils.parallel.LoadManager¶
Bases:
object
Keeps track of the amount of data managed by each process and helps keeping it balanced.
Note
A LoadManager should always be given the full range of ids, as the all-knowing power stems from the fact that it always knows how many processes there are, what rank it has and the full range of ids to distribute. Hence, please “assign” always the same list across all nodes.
- assign(idlist=None)¶
Subdivide the provided list of ids into contiguous blocks to balance the load among the processes.
The elements of idlist are used as keys to subsequently identify the rank of a given id, through the dict self.rank_of. No check is done whether ids passed are unique or even hashable (they better are)
If idlist is None, increase by one the load of the least busy process.
rank_of
is not updated in this case.- Parameters
idlist (list) – List of objects that can also be keys in a dict, i.e. hashable objects.
- Returns
R – A nested list, (a list of lists) such that
R[rank]=list
of elements of idlist managed by process of given rank.- Return type
list
- load¶
Represents the number of elements assigned to this process.
- rank_of¶
Rank of specific id, i.e. element of idlist.
- reset()¶
Resets
LoadManager
to initial state.
- ptypy.utils.parallel.MPInoise2d(sh, rms=1.0, mfs=2, rms_mod=None, mfs_mod=2)¶
Creates complex-valued statistical noise in the shape of sh consistent across all nodes.
- Parameters
sh (tuple) – Output shape.
rms (float or None) – Root mean square of noise in phase. If None, only ones are returned.
mfs (float) – Minimum feature size [in pixel] of noise in phase.
rms_mod (float or None) – Root mean square of noise in amplitude / modulus.
mfs_mod – Minimum feature size [in pixel] of noise in amplitude / modulus.
- Returns
noise – Numpy array filled with noise
- Return type
ndarray
See also
- ptypy.utils.parallel.MPIrand_normal(loc=0.0, scale=1.0, size=1)¶
Wrapper for
np.random.normal
for same random sample across all nodes. See numpy/scipy documentation below.normal(loc=0.0, scale=1.0, size=None)
Draw random samples from a normal (Gaussian) distribution.
The probability density function of the normal distribution, first derived by De Moivre and 200 years later by both Gauss and Laplace independently 2, is often called the bell curve because of its characteristic shape (see the example below).
The normal distributions occurs often in nature. For example, it describes the commonly occurring distribution of samples influenced by a large number of tiny, random disturbances, each with its own unique distribution 2.
Note
New code should use the
normal
method of adefault_rng()
instance instead; please see the random-quick-start.- Parameters
loc (float or array_like of floats) – Mean (“centre”) of the distribution.
scale (float or array_like of floats) – Standard deviation (spread or “width”) of the distribution. Must be non-negative.
size (int or tuple of ints, optional) – Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn. If size isNone
(default), a single value is returned ifloc
andscale
are both scalars. Otherwise,np.broadcast(loc, scale).size
samples are drawn.
- Returns
out – Drawn samples from the parameterized normal distribution.
- Return type
ndarray or scalar
See also
scipy.stats.norm
probability density function, distribution or cumulative density function, etc.
Generator.normal
which should be used for new code.
Notes
The probability density for the Gaussian distribution is
\[p(x) = \frac{1}{\sqrt{ 2 \pi \sigma^2 }} e^{ - \frac{ (x - \mu)^2 } {2 \sigma^2} },\]where \(\mu\) is the mean and \(\sigma\) the standard deviation. The square of the standard deviation, \(\sigma^2\), is called the variance.
The function has its peak at the mean, and its “spread” increases with the standard deviation (the function reaches 0.607 times its maximum at \(x + \sigma\) and \(x - \sigma\) 2). This implies that normal is more likely to return samples lying close to the mean, rather than those far away.
References
- 1
Wikipedia, “Normal distribution”, https://en.wikipedia.org/wiki/Normal_distribution
- 2(1,2,3)
P. R. Peebles Jr., “Central Limit Theorem” in “Probability, Random Variables and Random Signal Principles”, 4th ed., 2001, pp. 51, 51, 125.
Examples
Draw samples from the distribution:
>>> mu, sigma = 0, 0.1 # mean and standard deviation >>> s = np.random.normal(mu, sigma, 1000)
Verify the mean and the variance:
>>> abs(mu - np.mean(s)) 0.0 # may vary
>>> abs(sigma - np.std(s, ddof=1)) 0.1 # may vary
Display the histogram of the samples, along with the probability density function:
>>> import matplotlib.pyplot as plt >>> count, bins, ignored = plt.hist(s, 30, density=True) >>> plt.plot(bins, 1/(sigma * np.sqrt(2 * np.pi)) * ... np.exp( - (bins - mu)**2 / (2 * sigma**2) ), ... linewidth=2, color='r') >>> plt.show()
Two-by-four array of samples from N(3, 6.25):
>>> np.random.normal(3, 2.5, size=(2, 4)) array([[-4.49401501, 4.00950034, -1.81814867, 7.29718677], # random [ 0.39924804, 4.68456316, 4.99394529, 4.84057254]]) # random
- ptypy.utils.parallel.MPIrand_uniform(low=0.0, high=1.0, size=1)¶
Wrapper for
np.random.uniform
for same random sample across all nodes. See numpy/scipy documentation below.uniform(low=0.0, high=1.0, size=None)
Draw samples from a uniform distribution.
Samples are uniformly distributed over the half-open interval
[low, high)
(includes low, but excludes high). In other words, any value within the given interval is equally likely to be drawn by uniform.Note
New code should use the
uniform
method of adefault_rng()
instance instead; please see the random-quick-start.- Parameters
low (float or array_like of floats, optional) – Lower boundary of the output interval. All values generated will be greater than or equal to low. The default value is 0.
high (float or array_like of floats) – Upper boundary of the output interval. All values generated will be less than or equal to high. The high limit may be included in the returned array of floats due to floating-point rounding in the equation
low + (high-low) * random_sample()
. The default value is 1.0.size (int or tuple of ints, optional) – Output shape. If the given shape is, e.g.,
(m, n, k)
, thenm * n * k
samples are drawn. If size isNone
(default), a single value is returned iflow
andhigh
are both scalars. Otherwise,np.broadcast(low, high).size
samples are drawn.
- Returns
out – Drawn samples from the parameterized uniform distribution.
- Return type
ndarray or scalar
See also
randint
Discrete uniform distribution, yielding integers.
random_integers
Discrete uniform distribution over the closed interval
[low, high]
.random_sample
Floats uniformly distributed over
[0, 1)
.random
Alias for random_sample.
rand
Convenience function that accepts dimensions as input, e.g.,
rand(2,2)
would generate a 2-by-2 array of floats, uniformly distributed over[0, 1)
.Generator.uniform
which should be used for new code.
Notes
The probability density function of the uniform distribution is
\[p(x) = \frac{1}{b - a}\]anywhere within the interval
[a, b)
, and zero elsewhere.When
high
==low
, values oflow
will be returned. Ifhigh
<low
, the results are officially undefined and may eventually raise an error, i.e. do not rely on this function to behave when passed arguments satisfying that inequality condition. Thehigh
limit may be included in the returned array of floats due to floating-point rounding in the equationlow + (high-low) * random_sample()
. For example:>>> x = np.float32(5*0.99999999) >>> x 5.0
Examples
Draw samples from the distribution:
>>> s = np.random.uniform(-1,0,1000)
All values are within the given interval:
>>> np.all(s >= -1) True >>> np.all(s < 0) True
Display the histogram of the samples, along with the probability density function:
>>> import matplotlib.pyplot as plt >>> count, bins, ignored = plt.hist(s, 15, density=True) >>> plt.plot(bins, np.ones_like(bins), linewidth=2, color='r') >>> plt.show()
- ptypy.utils.parallel.allreduce(a, op=None)¶
Wrapper for comm.Allreduce, always in place.
- Parameters
a (numpy-ndarray) – The array to operate on.
op (operation) – MPI operation to execute. None or one of MPI.BAND, MPI.BOR, MPI.BXOR, MPI.LAND, MPI.LOR, MPI.LXOR, MPI.MAX, MPI.MAXLOC, MPI.MIN, MPI.MINLOC, MPI.OP_NULL, MPI.PROD, MPI.REPLACE or MPI.SUM. If None, uses MPI.SUM.
Note
Explanation : If process #1 has ndarray
a
and process #2 has ndarrayb
. After calling allreduce, the new arrays after allreduce area'=op(a,b)
andb'=op(a,b)
on process #1 and #2 respectively
- ptypy.utils.parallel.barrier()¶
Wrapper for comm.Barrier.
- ptypy.utils.parallel.bcast(data, source=0)¶
Wrapper for comm.bcast and comm.Bcast. If data is a numpy.ndarray, a header will be broadcasted first with comm.bcast that contains information on array shape and data type. Afterwards the array will be sent with comm.Bcast. If data is not a numpy.ndarray, the whole object will be pickled and broadcasted with comm.bcast in one go.
- Parameters
data (ndarray or other) – Object to send
source (int) – The rank of the source node / process. Defaults to 0 (master).
tag (int) – Defaults to 0.
- ptypy.utils.parallel.bcast_dict(dct, keys='all', source=0)¶
Broadcasts or scatters a dict dct from
rank==source
.- Parameters
keys (list or 'all') – List of keys whose values are accepted at each node. In case of
keys=all
, every node accepts all items andbcast_dict
acts as broadcast.source (int) – Rank of node / process which broadcasts / scatters.
- Returns
dct – A smaller dictionary with values to keys if that key was in source dictionary.
- Return type
dict
Note
There is no guarantee that each key,value pair is accepted at other nodes, except for
keys='all'
. Also in this implementation the input dct from source is completely transmitted to every node, potentially creating a large overhead for may nodes and huge dictionarysDeleting reference in input dictionary may result in data loss at
rank==source
See also
- ptypy.utils.parallel.gather_dict(dct, target=0)¶
Gathers broadcasted or scattered dict dct at rank target.
- Parameters
dct (dict) – Input dictionary. Remains unaltered
target (int) – Rank of process where the dct’s are gathered
- Returns
out – Gathered dict at
rank==target
, Empty dict atrank!=target
- Return type
dict
Note
If the same key exists on different nodes, the corresponding values will be consecutively overidden in the order of the ranks at the gathering node without complain or notification.
See also
- ptypy.utils.parallel.gather_list(lst, length, indices)¶
gathers list lst of all processes to a list of length length according to order given by indices. definitely not foolproof The user has to make sure that no index appears twice in all processes
return list of length length. has only meaning in master process
- ptypy.utils.parallel.receive(source=None, tag=0)¶
Wrapper for comm.Recv. Probes first with comm.recv. If the unpickled is a 3-tuple with first entry==’npy’, prepares buffer and waits with comm.Recv
- Parameters
source (int or None) – The rank of the node / process sending data. If None, this is set to MPI.ANY_SOURCE
tag (int) – Not really useful here - defaults to 0 all the time
- Returns
out
- Return type
ndarray or other
- ptypy.utils.parallel.send(data, dest=0, tag=0)¶
Wrapper for comm.Send and comm.send. If data is a numpy.ndarray, a header will be sent first with comm.send that contains information on array shape and data type. Afterwards the array will be sent with comm.Send. If data is not a numpy.ndarray, the whole object will be pickled and sent with comm.send in one go.
- Parameters
data (ndarray or other) – Object to send
dest (int) – The rank of the destination node / process. Defaults to 0 (master).
tag (int) – Defaults to 0.
ptypy.utils.parameters module¶
Parameter definition.
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- class ptypy.utils.parameters.Param(__d__=None, **kwargs)¶
Bases:
dict
Convenience class: a dictionary that gives access to its keys through attributes.
Note: dictionaries stored in this class are also automatically converted to Param objects: >>> p = Param() >>> p.x = {} >>> p Param({‘x’: {}})
While dict(p) returns a dictionary, it is not recursive, so it is better in this case to use p.todict(). However, p.todict does not check for infinite recursion. So please don’t store a dictionary (or a Param) inside itself.
BE: Please note also that the recursive behavior of the update function will create new references. This will lead inconsistency if other objects refer to dicts or Params in the updated Param instance.
A Dictionary that enables access to its keys as attributes. Same constructor as dict.
- __contains__(key)¶
Redefined to support dot keys.
- __dir__()¶
Defined to include the keys when using dir(). Useful for tab completion in e.g. ipython. If you do not wish the dict key’s be displayed as attributes (although they are still accessible as such) set the class attribute _display_items_as_attributes to False. Default is True.
- copy(depth=0)¶
- Returns Param
A (recursive) copy of P with depth depth
- update(__d__=None, in_place_depth=0, Convert=False, **kwargs)¶
Update Param - almost same behavior as dict.update, except that all dictionaries are converted to Param if Convert is set to True, and update may occur in-place recursively for other Param instances that self refers to.
- Parameters
Convert (bool) – If True, convert all dict-like values in self also to Param. WARNING This mey result in misdirected references in your environment
in_place_depth (int) – Counter for recursive in-place updates If the counter reaches zero, the Param to a key is replaced instead of updated
- ptypy.utils.parameters.asParam(obj)¶
Convert the input to a Param.
- Parameters
a (dict_like) – Input structure, in any format that can be converted to a Param.
Returns –
out (Param) – The Param structure built from a. No copy is done if the input is already a Param.
- ptypy.utils.parameters.param_from_json(filename)¶
Convert param tree from JSON file into Param().
- ptypy.utils.parameters.param_from_yaml(filename)¶
Convert param tree from YAML file into Param().
ptypy.utils.plot_client module¶
Client tools for plotting.
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- class ptypy.utils.plot_client.MPLClient(client_pars=None, autoplot_pars=None, home=None, layout_pars=None, in_thread=False, is_slave=False)¶
Bases:
ptypy.utils.plot_client.MPLplotter
Create a client and attempt to connect to a running reconstruction server.
- DEFAULT = {'dpi': 100, 'figsize': (16, 10), 'gridspecpars': (0.1, 0.12, 0.07, 0.95, 0.05, 0.93), 'home': '/tmp/ptypy/', 'interactive': True, 'movie': False, 'ob': {'auto_display': ['a', 'p'], 'clims': [None, [-3.141592653589793, 3.141592653589793]], 'cmaps': ['gray', 'hsv'], 'crop': [0.3, 0.3], 'layers': None, 'local_error': False, 'mask': 0.3, 'rm_pr': True, 'shape': None, 'use_colorbar': True}, 'plot_error': [True, True, True], 'pr': {'auto_display': ['c'], 'clims': [None, [-3.141592653589793, 3.141592653589793]], 'cmaps': ['gray', 'hsv'], 'crop': [0.3, 0.3], 'layers': None, 'local_error': False, 'mask': 0.3, 'rm_pr': True, 'shape': None, 'use_colorbar': True}, 'simplified_aspect_ratios': False}¶
- loop_plot()¶
Plot forever.
- class ptypy.utils.plot_client.MPLplotter(pars=None, probes=None, objects=None, runtime=None, in_thread=False)¶
Bases:
object
Plotting Client for Ptypy, using matplotlib.
Create a client and attempt to connect to a running reconstruction server.
- DEFAULT = {'dpi': 100, 'figsize': (16, 10), 'gridspecpars': (0.1, 0.12, 0.07, 0.95, 0.05, 0.93), 'home': '/tmp/ptypy/', 'interactive': True, 'movie': False, 'ob': {'auto_display': ['a', 'p'], 'clims': [None, [-3.141592653589793, 3.141592653589793]], 'cmaps': ['gray', 'hsv'], 'crop': [0.3, 0.3], 'layers': None, 'local_error': False, 'mask': 0.3, 'rm_pr': True, 'shape': None, 'use_colorbar': True}, 'plot_error': [True, True, True], 'pr': {'auto_display': ['c'], 'clims': [None, [-3.141592653589793, 3.141592653589793]], 'cmaps': ['gray', 'hsv'], 'crop': [0.3, 0.3], 'layers': None, 'local_error': False, 'mask': 0.3, 'rm_pr': True, 'shape': None, 'use_colorbar': True}, 'simplified_aspect_ratios': False}¶
- static create_plot_from_tile_list(fignum=1, num_shape_list=[(4, (2, 2))], figsize=(8, 8), dpi=100)¶
- draw()¶
- plot_all()¶
- plot_error()¶
- plot_storage(storage, plot_pars, title='', typ='obj')¶
- save(pattern, count=0)¶
- update_plot_layout()¶
Generate the plot layout.
- class ptypy.utils.plot_client.PlotClient(client_pars=None, in_thread=False)¶
Bases:
object
A client that connects and continually gets the data required for plotting. Note: all data is transferred as soon as the server provides it. This might be a waste of bandwidth if all that is required is a client that plots “on demand”…
This PlotClient doesn’t actually plot.
Create a client and attempt to connect to a running reconstruction server.
- ACTIVE = 1¶
- DATA = 2¶
- STOPPED = 0¶
- disconnect()¶
Disconnect from reconstruction server.
- get_data()¶
Thread-safe way to copy data buffer. :return:
- start()¶
This needs to be run for the thread to initialize.
- property status¶
True only if new data were acquired since last time checked.
- stop()¶
Stop loop plot
- ptypy.utils.plot_client.figure_from_ptycho(P, pars=None)¶
Returns a matplotlib figure displaying a reconstruction from a Ptycho instance.
- Parameters
P (Ptycho) – Ptycho instance
pars (Plotting paramters, optional) – plotting template as u.Param() istance
- Returns
fig
- Return type
matplotlib.figure.Figure
- ptypy.utils.plot_client.figure_from_ptyr(filename, pars=None)¶
Returns a matplotlib figure displaying a reconstruction from a .ptyr file.
- Parameters
filename (str) – path to .ptyr file
pars (Plotting paramters, optional) – plotting template as u.Param() istance
- Returns
fig
- Return type
matplotlib.figure.Figure
- ptypy.utils.plot_client.spawn_MPLClient(client_pars, autoplot_pars, home=None)¶
A function that creates and runs a silent instance of MPLClient.
ptypy.utils.plot_utils module¶
Plotting utilities.
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- class ptypy.utils.plot_utils.PtyAxis(ax=None, data=None, channel='r', cmap=None, fontsize=8, **kwargs)¶
Bases:
object
Plot environment for matplotlib to allow for a Image plot with color axis, potentially of a potentially complex array.
Please note that this class may undergo changes or become obsolete altogether.
- Parameters
ax (pyplot.axis) – An axis in matplotlib figure. If
None
a figure with a single axis will be created.data (numpy.ndarray) – The (complex) twodimensional data to be displayed.
channel (str) –
Choose
'a'
to plot absolute/modulus value of the data,'p'
to plot phase value of the data,'a'
to plot real value of the data,'a'
to plot imaginary value of the data,'a'
to plot a composite image where phase channel maps to hue and modulus channel maps to brightness of the color.
cmap (str) – String representation of one of matplotlibs colormaps.
fontsize (int) – Base font size of labels, etc.
- Keyword Arguments
vmax (vmin,) – Minimum and maximum value for colormap scaling
rmramp (bool) – Remove phase ramp if
True
, default isFalse
- add_colorbar(aspect=10, fraction=0.2, pad=0.02, resolution=256)¶
- set_channel(channel, update=True)¶
- set_clims(vmin, vmax, update=True)¶
- set_cmap(cmap, update=True)¶
- set_data(data, update=True)¶
- set_mask(mask, update=True)¶
- set_psize(psize, update=True)¶
- ptypy.utils.plot_utils.complex2hsv(cin, vmin=0.0, vmax=None)¶
Transforms a complex array into an RGB image, mapping phase to hue, amplitude to value and keeping maximum saturation.
- Parameters
cin (ndarray) – Complex input. Must be two-dimensional.
vmin (float) – Clip amplitude of input into this interval.
vmax (float) – Clip amplitude of input into this interval.
- Returns
rgb – Three dimensional output.
- Return type
ndarray
See also
- ptypy.utils.plot_utils.complex2rgb(cin, **kwargs)¶
Executes complex2hsv and then hsv2rgb
See also
- ptypy.utils.plot_utils.franzmap()¶
Set the default colormap to Franz’s map and apply to current image if any.
- ptypy.utils.plot_utils.hsv2complex(cin)¶
Reverse to
complex2hsv
- ptypy.utils.plot_utils.hsv2rgb(hsv)¶
HSV (Hue,Saturation,Value) to RGB (Red,Green,Blue) transformation.
- Parameters
hsv (array-like) – Input must be two-dimensional. First axis is interpreted as hue,saturation,value channels.
- Returns
rgb – Three dimensional output. Last axis is interpreted as red, green, blue channels.
- Return type
ndarray
See also
- ptypy.utils.plot_utils.imload(filename)¶
Load an image and returns a numpy array. May get deleted
- ptypy.utils.plot_utils.imsave(a, filename=None, vmin=None, vmax=None, cmap=None)¶
Take array a and transform to PIL.Image object that may be used by pyplot.imshow for example. Also save image buffer directly without the sometimes unnecessary Gui-frame and overhead.
- Parameters
a (ndarray) – Two dimensional array. Can be complex, in which case the amplitude will be optionally clipped by vmin and vmax if set.
filename (str, optionsl) – File path to save the image buffer to. Use ‘*.png’ or ‘*.png’ as image formats.
vmin (float, optional) – Value limits (‘clipping’) to fit the color scale. If not set, color scale will span from minimum to maximum value in array
vmax (float, optional) – Value limits (‘clipping’) to fit the color scale. If not set, color scale will span from minimum to maximum value in array
cmap (str, optional) – Name of the colormap for colorencoding.
- Returns
im – a PIL.Image object.
- Return type
PIL.Image
See also
Examples
>>> from ptypy.utils import imsave >>> from matplotlib import pyplot as plt >>> from ptypy.resources import flower_obj >>> a = flower_obj(512) >>> pil = imsave(a) >>> plt.imshow(pil) >>> plt.show()
converts array a into, and returns a PIL image and displays it.
>>> pil = imsave(a, /tmp/moon.png)
returns the image and also saves it to filename
>>> imsave(a, vmin=0, vmax=0.5)
clips the array to values between 0 and 0.5.
>>> imsave(abs(a), cmap='gray')
uses a matplotlib colormap with name ‘gray’
- ptypy.utils.plot_utils.pause(interval)¶
Run the GUI event loop for interval seconds.
If there is an active figure, it will be updated and displayed before the pause, and the GUI event loop (if any) will run during the pause.
This can be used for crude animation. For more complex animation use
matplotlib.animation
.If there is no active figure, sleep for interval seconds instead.
See also
matplotlib.animation
Proper animations
show
Show all figures and optional block until all figures are closed.
- ptypy.utils.plot_utils.plot_storage(S, fignum=100, modulus='linear', slices=(slice(None, 1, None), slice(None, None, None), slice(None, None, None)), si_axes='x', mask=None, **kwargs)¶
Quickly display the data buffer of a
Storage
instance.Keyword arguments are the same as
PtyAxis
- Parameters
S (Storage) – Storage instance
fignum (int, optional) – Number of the figure.
slices (tuple of slices or string of numpy index expression, optional) – Determines what part of Storage buffer is displayed, i.e. which layers and which region-of-interest. Layers are subplotted horizontically next to each other. Figsize is (6,6*layers)
modulus (str, optional) – One of sqrt, log or linear to apply to modulus of array buffer. Useful to reduce dynamic range for diffraction images.
si_axes (str, optional) – One of ‘x’,’xy’,’y’ or None, determins which axes display length units instead of pixel units
mask (ndarray, scalar or None) – Bool array of valid pixel data for rescaling.
- Returns
fig
- Return type
maplotlib.pyplot.figure
- ptypy.utils.plot_utils.rgb2complex(rgb)¶
Reverse to
complex2rgb
- ptypy.utils.plot_utils.rmphaseramp(a, weight=None, return_phaseramp=False)¶
Attempts to remove the phase ramp in a two-dimensional complex array
a
.- Parameters
a (ndarray) – Input image as complex 2D-array.
weight (ndarray, str, optional) – Pass weighting array or use
'abs'
for a modulus-weighted phaseramp andNon
for no weights.return_phaseramp (bool, optional) – Use True to get also the phaseramp array
p
.
- Returns
out (ndarray) – Modified 2D-array,
out=a*p
p (ndarray, optional) – Phaseramp if
return_phaseramp = True
, otherwise omitted
Examples
>>> b = rmphaseramp(image) >>> b, p = rmphaseramp(image , return_phaseramp=True)
ptypy.utils.scripts module¶
Longer script-like functions.
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- ptypy.utils.scripts.cxro_iref(formula, energy, density=- 1, npts=100)¶
Query CXRO database for index of refraction values for a solid.
- Parameters
formula (str) – String representation of the Formula to use.
energy (float or (float, float)) – Either a single energy (in eV) or the minimum/maximum bounds.
density (None or float, optional) – Density of the material [g/ccm]. If
None
or<0
the regular density at ambient temperatures is used.npts (int, optional) – Number of points between the min and max energies.
- Returns
energy, delta, beta – Energy used and the respective delta and beta values.
- Return type
scalar or vector
- ptypy.utils.scripts.diversify(A, noise=None, shift=None, power=1.0)¶
Add diversity to 3d numpy array A, acts in-place.
- Parameters
A (np.array) – Input numpy array.
noise (2-tuple or 4-tuple) – For detailed description see
ptypy.utils.parallel.MPInoise2d
power (float, tuple) – Relative power of layers with respect to the first (0) layer. Can be scalar or tuple / array
shift (float, tuple) – Relative shift of layers with respect to the first (0) layer. Can be scalar or tuple / array not implemented yet
See also
- ptypy.utils.scripts.hdr_image(img_list, exp_list, thresholds=[3000, 50000], dark_list=[], avg_type='highest', mask_list=[], ClipLongestExposure=False, ClipShortestExposure=False)¶
Generate a high dynamic range image from a list of images img_list and corresponding exposure information in exp_list.
- Parameters
img_list (list) – Sequence of images (as 2d np.ndarray)
exp_list (list of float) – Associated exposures to each element of above sequence img_list
thresholds (list, 2-tuple) – Tuple of lower limit (noise floor) and upper limit (overexposure) in the images.
dark_list (list) – Single frame or sequence of dark images (as 2d np.array) of the same length as img_list. These frames are used for dark-field correction
avg_type (str) –
Type of combining all valid pixels:
’highest’, the next longest exposure is used to replace overexposed pixels.
<other_string>, overexposed pixels are replaced by the pixel average of all other images with valid pixel values for that pixel.
mask_list (list) – Single frame or sequence of 2d np.array. Provide additional masking (dead pixels, hot pixels)
ClipLongestExposure (bool) – If True, also mask the noise floor in the longest exposure.
ClipShortestExposure (bool) – if True, also mask the overexposed pixels in the shortest exposure.
- Returns
out – Combined hdr-image with shape of one of the frame in img_list
- Return type
ndarray
Examples
>>> from ptypy import io >>> dark_list, meta = io.image_read('/path/to/dark/images/ccd*.raw') >>> img_list, meta= io.image_read('/path/to/images/ccd*.raw') >>> exp_list = [meta[j]['exposure_time__key'] for j in range(len(meta))] >>> hdr, masks = hdr_image(img_list, exp_list, dark_list=dark_list)
- ptypy.utils.scripts.load_from_ptyr(filename, what='probe', ID=None, layer=None)¶
Convenience script to extract data from
*.ptyr
-file.- Parameters
filename (str) – Full Path to a
*.ptyr
data file. No check on the file suffix is done. Any compatible hdf5 formatted file is allowed.what (str) – Type of container to retrieve. Only ‘probe’ and ‘obj’ makes sense. Default is ‘probe’
ID (str) – ID of storage in chosen container. If
None
the first stored storage is chosen.layer (int, optional) – If an integer, the data buffer of chosen storage gets sliced with layer for its first index.
- Returns
data – If layer is provided, that layer
storage,data[layer]
will be sliced from the 3d data buffer, else the whole bufferstorage.data
will be returned.- Return type
ndarray
- ptypy.utils.scripts.mass_center(A, axes=None, mask=None)¶
Calculates mass center of n-dimensional array A along tuple of axis axes.
- Parameters
A (ndarray) – input array
axes (list, tuple) – Sequence of axes that contribute to distributed mass. If
axes==None
, all axes are considered.mask (ndarray, None) – If not None, an array the same shape as A acting as mask
- Returns
mass – Center of mass in pixel for each axis selected.
- Return type
1d array
- ptypy.utils.scripts.phase_from_dpc(dpc_row, dpc_col)¶
Implements fourier integration method for two differential quantities.
Assumes 2 arrays of N-dimensions which contain differential quantities in X and Y, i.e. the LAST two dimensions (-2 & -1) in this case.
- Parameters
dpc_row (ndarray) – Differential information along 2nd last dimension and last dimension respectively. Must be of the same shape.
dpc_col (ndarray) – Differential information along 2nd last dimension and last dimension respectively. Must be of the same shape.
- Returns
out – Integrated array of same shape as dpc_row and dpc_col.
- Return type
ndarray
- ptypy.utils.scripts.png2mpg(listoffiles, framefile='frames.txt', fps=5, bitrate=2000, codec='wmv2', Encode=True, RemoveImages=False)¶
Makes a movie (*.mpg) from a collection of *.png or *.jpeg frames. Requires binary of mencoder installed on system
- Parameters
listoffiles (list of str) – A list of paths to files. Each file will be reinterpreted as a collection files in the same directory, e.g. only the first file in a series needs to be selected. All series for which a first file was given will be concatenated in the movie. May also be a textfile with a listing of frames in which case the creation of an intermediate file of frame paths will be skipped.
framefile (str) – Filepath, the respective file will be created to store a list of all frames. This file will be used by mencoder later.
fps (scalar) – Frames per second in final movie
bitrate (int) – Encoding detail, determines video quality
codec (str) – Defines the codec to Use
Encode (bool) – If True, video will be encoded calling mencoder If False, mencoder will not be called, but the necessary command expression will be returned instead. Very well suited for a dry-run
RemoveImages (bool) – If True, all images referred to by framefile are deleted except for the last frame.
- Returns
cmd – Command string for the shell to encode the video later manually.
- Return type
str
Examples
>>> from ptypy.utils.scripts import png2mpg >>> png2mpg(['/path/to/image_000.png'])
The following happens: 1) search for files similar to image_*.png in ‘/path/to/’ 2) found files get listed in a file ‘/path/to/frames.txt’ 3) calls mencoder to use that file to encode a movie with the default args. 4) movie is in the same folder as ‘frames.txt’ and is called ‘frames.mpg’
>>> png2mpg(['/path1/to/imageA_040.png', '/path2/to/imageB_001.png'], >>> framefile='./banana.txt')
Generates list file ‘banana_text’ in current folder. The list file contains in order every path compatible with the wildcards ‘/path1/to/imageA_*.png’ and ‘/path2/to/imageB_*.png’
>>> str = png2mpg(['/path/to/image_000.png'], Encode=False)
Returns encoder argument string. Use os.system(encoderstring) for encoding manually later
- ptypy.utils.scripts.radial_distribution(A, radii=None)¶
Returns radial mass distribution up to radii in radii.
- Parameters
A (ndarray) – input array
radii (list, tuple) – Sequence of radii to calculate enclosed mass. If None, the sequence defaults to
range(1, np.min(A.shape) / 2)
- Returns
radii, masses – Sequence of used radii and corresponding integrated mass masses. Sequences have the same length.
- Return type
list
- ptypy.utils.scripts.remove_hot_pixels(data, size=3, tolerance=3, ignore_edges=False)¶
Replaces outlier data points with the median of the surrounding data points.
Removes outlier data points of a 2D numpy array and replaces them with the median of the surrounding data points. Original code (see at the end of function) had been published by DanHickstein on stackoverflow.com under CC BY-SA license.
- Parameters
data (2d np.array) – Input 2D numpy array to be corrected.
size (int) – Size of the window on which the median filter will be applied around every data point.
tolerance (int) – Tolerance multiplied with the standard deviation of the data array subtracted by the blurred array (difference array) yields the threshold for cutoff.
ignore_edges (bool) – If True, edges of the array are ignored, which speeds up the code.
- Returns
out – Returns a 2d np.array where outlier data points have been replaced with the median of the surrounding data points and a list containing the coordinates of the outlier data points.
- Return type
2d array, list
- ptypy.utils.scripts.stxm_analysis(storage, probe=None)¶
Performs a stxm analysis on a storage using the pods.
This function is MPI compatible.
- Parameters
storage (ptypy.core.Storage instance) – A
Storage
instance to be analysedprobe (None, scalar or array) –
If None, picks a probe from the first view’s pod
If scalar, uses a Gaussian with probe as standard deviation
Else: attempts to use passed value directly as 2d-probe
- Returns
trans (ndarray) – Transmission of shape
storage.shape
.dpc_row (ndarray) – Differential phase contrast along row-coordinates, i.e. vertical direction (y-direction) of shape
storage.shape
.dpc_col (ndarray) – Differential phase contrast along column-coordinates, i.e. horizontal direction (x-direction) of shape
storage.shape
.
- ptypy.utils.scripts.stxm_init(storage, probe=None)¶
Convenience script that performs a STXM analysis for storage storage given a probe array probe and stores result back in storage.
See also
- ptypy.utils.scripts.xradia_star(sh, spokes=48, std=0.5, minfeature=5, ringfact=2, rings=4, contrast=1.0, Fast=False)¶
Creates an Xradia-like star pattern on an array of shape sh.
Works superb as test pattern in ptychography.
requires scipy
- Parameters
sh (int, tuple) – Shape of requested array.
std (float) –
- “Resolution” of xradia star, i.e. standard deviation of the
error function used for smoothing the edges (in pixel).
spokes (int, optional) – Number of spokes.
minfeature (float, optional) – Spoke width at the smallest (inner) tip (in pixel).
ringfact (float) – Increase in feature size from ring to ring (factor). Determines position of the rings.
rings (int) – Number of rings with spokes.
contrast (float) – Minimum contrast, set to zero for a gradual increase of the profile from zero to 1, set to 1 for no gradient in profile.
Fast (bool) – If set to False, the error function is evaluated at the edges. Preferred choice when using fft, as edges are less prone to antialiasing in this case. If set to True, simple boolean comparison will be used instead to draw the edges and the result is later smoothed with a gaussian filter. Preferred choice for impatient users, as this choice is roughly a factor 2 faster for larger arrays
Examples
>>> from ptypy.utils.scripts import xradia_star >>> # Base configuration >>> X1 = xradia_star(1024) >>> # Few spokes single ring >>> X2 = xradia_star(1024, 12, std=4, rings=1, minfeature=10, ringfact=10) >>> # Very fine plus gradient >>> X3 = xradia_star(1024, 64, std = 0.2, rings = 10, minfeature=1, >>> contrast=0) >>> from matplotlib import pyplot as plt >>> ax = plt.subplot(131) >>> ax.imshow(X1, cmap='gray') >>> ax = plt.subplot(132) >>> ax.imshow(X2, cmap='gray') >>> ax = plt.subplot(133) >>> ax.imshow(X3, cmap='gray') >>> plt.show()
ptypy.utils.descriptor module¶
Parameter validation. This module parses the file
resources/parameters_descriptions.csv
to extract the parameter
defaults for PtyPy. It saves all parameters in the form of
a PDesc
object, which are flat listed in
parameter_descriptions or in entry_points_dct, which only contains
parameters with subparameters (children).
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- class ptypy.utils.descriptor.ArgParseDescriptor(name, parent=None, separator='.')¶
Bases:
ptypy.utils.descriptor.Descriptor
- Parameters
name (str) – The name of the parameter represented by this instance
parent (Descriptor or None) – Parent parameter or None if no parent parameter.
separator (str) – Subtree separator character. Defaults to ‘.’.
- OPTIONS_DEF = {'choices': 'If parameter is list of choices, these are listed here.', 'default': 'Default value for parameter.', 'help': 'A small docstring for command line parsing (required).'}¶
- add2argparser(parser=None, prefix='', excludes=('scans', 'engines'), mode='add')¶
Add parameter to an argparse.ArgumentParser instance (or create and return one if parser is None) prefix is
- property choices¶
If parameter is a list of choices, these are listed here.
- property default¶
Returns default as a Python type
- eval(val)¶
A more verbose wrapper around ast.literal_eval
- property help¶
Short descriptive explanation of parameter
- class ptypy.utils.descriptor.Descriptor(name, parent=None, separator='.')¶
Bases:
object
Base class for parameter descriptions and validation. This class is used to hold both command line arguments and Param-type parameter descriptions.
- Variables
OPTIONS_DEF – A dictionary whose keys are attribute names and values are description of the attribute. It this description contains the text “required” or “mandatory”, the attribute is registered as required.
- Parameters
name (str) – The name of the parameter represented by this instance
parent (Descriptor or None) – Parent parameter or None if no parent parameter.
separator (str) – Subtree separator character. Defaults to ‘.’.
- OPTIONS_DEF = None¶
- __getitem__(name)¶
Get a descendant
- __setitem__(name, desc)¶
Insert a descendant
- __str__()¶
Pretty-print the Parameter options in ConfigParser format.
- add_child(desc, copy=False)¶
- children¶
Hierarchical tree of sub-Parameters.
- property descendants¶
Iterator over all descendants as a pair (path name, object).
- from_string(s, strict=False, **kwargs)¶
Load Parameter from string using Python’s ConfigParser
Each parameter occupies its own section. Separator characters in sections names map to a tree-hierarchy.
Keyword arguments are forwarded to ConfigParser.RawConfigParser
- get(path)¶
return self.root[path] if it exists, None otherwise.
- property is_child¶
Type check
- load_conf_parser(fbuffer, **kwargs)¶
Load Parameter defaults using Python’s ConfigParser
Each parameter occupies its own section. Separator characters in sections names map to a tree-hierarchy.
Keyword arguments are forwarded to ConfigParser.RawConfigParser
- load_csv(fbuffer, **kwargs)¶
Load from csv as a fielded array. Keyword arguments are passed on to csv.DictReader
- load_json(fbuffer)¶
- name¶
Name of parameter
- new_child(name, options=None, implicit=False)¶
Create a new descendant and pass new options.
If name contains separators, intermediate children are created.
If name already exists, update options and return existing child.
If name already exists and had been created implicitly to create a child further down, the order in self.children is corrected to honor the order of explicitly created children. This behaviour can be deactivated by setting implicit=True.
- property option_keys¶
- options¶
Attributes to the parameters.
- parent¶
Parent parameter (
Descriptor
type) if it has one.
- property path¶
Return complete path from root of parameter tree. (self.root[self.path] == self should always be True unless self.root is root)
- prune_child(name)¶
Remove and return the parameter “name” and all its children. :param name: The descendant name :return: The Parameter object.
- property root¶
Return root of parameter tree.
- save_conf_parser(fbuffer, print_optional=True)¶
Save Parameter defaults using Pythons ConfigParser
Each parameter occupies its own section. Separator characters in sections names map to a tree-hierarchy.
- save_csv(fbuffer, **kwargs)¶
Save to fbuffer. Keyword arguments are passed on to csv.DictWriter
- save_json(fbuffer)¶
- to_string()¶
Return the full content of descriptor as a string in configparser format.
- class ptypy.utils.descriptor.EvalDescriptor(name, parent=None, separator='.')¶
Bases:
ptypy.utils.descriptor.ArgParseDescriptor
Parameter class to store metadata for all ptypy parameters (default, limits, documentation, etc.)
Parameter class to store metadata for all ptypy parameters (default, limits, documentation, etc.)
- OPTIONS_DEF = {'choices': 'If parameter is list of choices, these are listed here.', 'default': 'Default value for parameter (required).', 'doc': 'A longer explanation for the online docs.', 'help': 'A small docstring for command line parsing (required).', 'lowlim': 'Lower limit for scalar / integer values', 'type': 'Comma separated list of acceptable types.', 'uplim': 'Upper limit for scalar / integer values', 'userlevel': 'User level, a higher level means a parameter that is \n less likely to vary or harder to understand.'}¶
- check(pars, depth=99)¶
Check that input parameter pars is consistent with parameter description, up to given depth.
- Parameters
pars (The input parameter or parameter tree) –
depth (The level at wich verification is done.) –
- Return type
A dictionary report using CODES values.
- create_template(filename=None, start_at_root=True, user_level=0, doc_level=2)¶
Creates templates for ptypy scripts from an EvalDescriptor instance.
- property default¶
Default value as a Python type
- property doc¶
Long documentation, may contain sphinx inline markup.
- property is_evaluable¶
- property is_symlink¶
True if type/default are symlinks.
- property is_target¶
True if parent of symlink targets.
- property limits¶
(lower, upper) limits if applicable. (None, None) otherwise
- make_default(depth=0)¶
Creates a default parameter structure.
- Parameters
depth (int) – The depth in the structure to which all sub nodes are expanded All nodes beyond depth will be ignored.
- Returns
pars – A parameter branch as Param.
- Return type
Examples
>>> from ptypy.utils.descriptor import defaults_tree >>> print(defaults_tree['io'].make_default(depth=5))
- make_doc_rst(prst, use_root=True)¶
Pretty-print in RST format the whole structure.
- parse_doc(name=None, recursive=True)¶
Decorator to parse docstring and automatically attach new parameters. The parameter section is identified by a line starting with the word “Parameters”
- Parameters
name (str) – The descendant name under which all parameters will be held. If None, use self.
recursive (bool) – Whether or not to traverse the docstring of base classes. Is there are use case for this?
- Return type
The decorator function.
- sanity_check(depth=10)¶
Checks if default parameters from configuration are self-constistent with limits and choices.
- property type¶
List of possible data types.
- property userlevel¶
User level, a higher level means a parameter that is less likely to vary or harder to understand.
- validate(pars, raisecodes=(0, 4))¶
Check that the parameter structure pars matches the documented constraints for this node / parameter.
The function raises a RuntimeError if one of the code in the list raisecodes has been found. If raisecode is empty, the function will always return successfully but problems will be logged using logger.
- Parameters
pars (Param, dict) – A parameter set to validate
raisecodes (list) – List of codes that will raise a RuntimeError.
ptypy.utils.verbose module¶
Verbose package, based on the standard logging library.
Use as: from verbose import logger logger.warning(‘This is a warning’) logger.info(‘This is an information’) …
TODO: - Handlers for file and for interaction server - option for logging.disable to reduce call overhead
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.
- ptypy.utils.verbose.log(level, msg, parallel=False)¶
- ptypy.utils.verbose.report(thing, depth=4, noheader=False)¶
no protection for circular references
- ptypy.utils.verbose.set_level(level)¶
Set verbosity level. Kept here for backward compatibility
Module contents¶
Util sub-package
This file is part of the PTYPY package.
- copyright
Copyright 2014 by the PTYPY team, see AUTHORS.
- license
see LICENSE for details.