ptypy.experiment package

Submodules

ptypy.experiment.ID16Anfp module

Scan loading recipe for the ID16A beamline at ESRF - near-field ptycho setup.

This file is part of the PTYPY package.

copyright

Copyright 2014 by the PTYPY team, see AUTHORS.

license

see LICENSE for details.

class ptypy.experiment.ID16Anfp.ID16AScan(pars=None, **kwargs)

Bases: ptypy.core.data.PtyScan

Subclass of PtyScan for ID16A beamline (specifically for near-field ptychography).

Default data parameters. See scan.data

Create a PtyScan object that will load ID16A data.

DEFAULT = {'add_poisson_noise': False, 'auto_center': False, 'base_path': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'dark_subtraction': False, 'det_flat_field': None, 'dfile': None, 'distance': 7.19, 'distortion_h_file': '/data/id16a/inhouse1/instrument/img1/optique_peter_distortion/detector_distortion2d_v.edf', 'distortion_v_file': '/data/id16a/inhouse1/instrument/img1/optique_peter_distortion/detector_distortion2d_h.edf', 'energy': 7.2, 'experimentID': None, 'flat_division': False, 'label': None, 'load_parallel': 'data', 'mask_file': None, 'min_frames': 1, 'motors': ['spy', 'spz'], 'motors_multiplier': 1e-06, 'name': 'ID16AScan', 'num_frames': None, 'orientation': (False, True, False), 'positions_theory': None, 'psize': 0.000172, 'rebin': None, 'recipe': {}, 'sample_name': None, 'save': None, 'shape': 256, 'version': 0.1, 'whitefield_file': '/data/id16a/inhouse1/instrument/whitefield/white.edf'}
check(frames, start=0)

This method checks how many frames the preparation routine may process, starting from frame start at a request of frames. Returns the number of frames available from starting index start, and whether the end of the scan was reached.

Parameters

  • frames (int or None) – Number of frames requested

  • start (int or None) – Scanpoint index to start checking from

Returns

  • frame_accessible (int) – Number of frames readable from a starting point start

  • end_of_scan (bool or None) – Check if the end of scan was reached, otherwise None if this routine doesn’t know

correct(raw, weights, common)

Place holder for dark and flatfield correction. Apply (eventual) corrections to the raw frames. Convert from “raw” frames to usable data.

Parameters

  • raw (dict) – Dict containing index matched data frames (np.array).

  • weights (dict) – Dict containing possible weights.

  • common (dict) – Dict containing possible dark and flat frames.

Returns

data, weights – Flat and dark-corrected data dictionaries. These dictionaries must have the same keys as the input raw and contain corrected frames (data) and statistical weights (weights)

Return type

dict

Note

If the negative values results from the calculation, they will be forced to be equal to 0.

load(indices)

Loads data according to node specific scanpoint indices that have been determined by LoadManager or otherwise.

Returns

raw, pos, weight – Dictionaries whose keys are the given scan point indices and whose values are the respective frame / position according to the scan point index. weight and positions may be empty

Return type

dict

Note

If one weight (mask) is to be used for the whole scan, it should be loaded with load_weights(). The same goes for the positions, which sould be loade with load_positions().

load_common()

Loads anything that is common to all frames and stores it in dict.

Returns

common – contains information common to all frames such as dark, flat-field, detector flat-field, normalization couter, and distortion files

Return type

dict

load_positions()

Loads all positions for all diffraction patterns in this scan.

Returns

Positions – A (N,2)-array where N is the number of positions.

Return type

ndarray

load_weight()

Function description see parent class. For now, this function will be used to load the mask.

Returns

weight2d – A two-dimensional array with a shape compatible to the raw diffraction data frames if provided from file

Return type

ndarray

ptypy.experiment.ID16Anfp.undistort(frame, delta)

Frame distortion correction (linear interpolation) Any value outside the frame is replaced with a constant value (mean of the complete frame)

Parameters

  • frame (ndarray) – The input frame data

  • delta (2-tuple) – Containing the horizontal and vertical displacements respectively.

Returns

outf – The corrected frame of same dimension and type as frame.

Return type

ndarray

ptypy.experiment.hdf5_loader module

Scan loading recipe for the Diamond beamlines.

This file is part of the PTYPY package.

copyright

Copyright 2014 by the PTYPY team, see AUTHORS.

license

see LICENSE for details.

class ptypy.experiment.hdf5_loader.Hdf5Loader(pars=None, swmr=False, **kwargs)

Bases: ptypy.core.data.PtyScan

First attempt to make a generalised hdf5 loader for data. Please raise a ticket in github if changes are required so we can coordinate. There will be a Nexus and CXI subclass to this in the future.

hdf5 data loader

DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'darkfield': {'file': None, 'key': None}, 'dfile': None, 'distance': 7.19, 'electron_data': False, 'energy': 7.2, 'experimentID': None, 'flatfield': {'file': None, 'key': None}, 'framefilter': {'file': None, 'key': None}, 'frameorder': {'indices': None}, 'intensities': {'file': None, 'key': None}, 'label': None, 'load_parallel': 'data', 'mask': {'file': None, 'invert': False, 'key': None}, 'min_frames': 1, 'name': 'Hdf5Loader', 'normalisation': {'file': None, 'key': None, 'sigma': 3}, 'num_frames': None, 'orientation': None, 'outer_index': None, 'padding': None, 'positions': {'bounding_box': {'fast_axis_bounds': None, 'slow_axis_bounds': None}, 'fast_index': None, 'fast_key': None, 'fast_multiplier': 1.0, 'file': None, 'skip': 1, 'slow_index': None, 'slow_key': None, 'slow_multiplier': 1.0}, 'positions_theory': None, 'psize': 0.000172, 'rebin': None, 'recorded_distance': {'file': None, 'key': None, 'multiplier': 1.0}, 'recorded_energy': {'file': None, 'key': None, 'multiplier': 1.0, 'offset': 0.0}, 'recorded_psize': {'file': None, 'key': None, 'multiplier': 1.0}, 'save': None, 'shape': None, 'version': 0.1}
compute_scan_mapping_and_trajectory(data_shape, positions_fast_shape, positions_slow_shape)

This horrendous block of logic is all to do with making a semi-intelligent guess at what the data looks like.

get_corrected_intensities(index)

Corrects the intensities for darkfield, flatfield and normalisations if they exist. There is a lot of logic here, I wonder if there is a better way to get rid of it. Limited a bit by the MPI, and thinking about extension to large data size.

load_mapped_and_arbitrary_scan(indices)
load_mapped_and_raster_scan(indices)
load_unmapped_raster_scan(indices)
subtract_dark(raw, dark)

Subtract dark current from a raw frame and truncate negative values

class ptypy.experiment.hdf5_loader.Hdf5LoaderFast(pars=None, **kwargs)

Bases: ptypy.experiment.hdf5_loader.Hdf5Loader

hdf5 data loader

DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'darkfield': {'file': None, 'key': None}, 'dfile': None, 'distance': 7.19, 'electron_data': False, 'energy': 7.2, 'experimentID': None, 'flatfield': {'file': None, 'key': None}, 'framefilter': {'file': None, 'key': None}, 'frameorder': {'indices': None}, 'intensities': {'file': None, 'key': None}, 'label': None, 'load_parallel': 'data', 'mask': {'file': None, 'invert': False, 'key': None}, 'min_frames': 1, 'name': 'Hdf5Loader', 'normalisation': {'file': None, 'key': None, 'sigma': 3}, 'num_frames': None, 'orientation': None, 'outer_index': None, 'padding': None, 'positions': {'bounding_box': {'fast_axis_bounds': None, 'slow_axis_bounds': None}, 'fast_index': None, 'fast_key': None, 'fast_multiplier': 1.0, 'file': None, 'skip': 1, 'slow_index': None, 'slow_key': None, 'slow_multiplier': 1.0}, 'positions_theory': None, 'psize': 0.000172, 'rebin': None, 'recorded_distance': {'file': None, 'key': None, 'multiplier': 1.0}, 'recorded_energy': {'file': None, 'key': None, 'multiplier': 1.0, 'offset': 0.0}, 'recorded_psize': {'file': None, 'key': None, 'multiplier': 1.0}, 'save': None, 'shape': None, 'version': 0.1}
get_corrected_intensities(weights, intensities, index, indexed_frame_slice)

Corrects the intensities for normalisation and padding

load_mapped_and_arbitrary_scan(indices)
load_mapped_and_raster_scan(indices)
load_multiprocessing(src_slices)
load_unmapped_raster_scan(indices)
static subtract_dark(raw, dark)

Subtract dark current from a raw frame and truncate negative values

ptypy.experiment.optiklabor module

Created on Nov 22 2013

@author: Bjeorn Enders

class ptypy.experiment.optiklabor.FliSpecScanMultexp(pars=None, **kwargs)

Bases: ptypy.core.data.PtyScan

Class creation with minimum set of parameters, see PtyScan.DEFAULT Please note that class creation is not meant to load data.

Call initialize to begin loading and data file creation.

DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'base_path': '/data/CDI/opticslab_sxdm_2013/', 'center': 'auto', 'chunk_format': '.chunk%02d', 'dark_dir_pattern': '%(base_path)s%(scan_dir)sS%(dark_number)05d/', 'dark_number': 72, 'data_dir_pattern': '%(base_path)s%(scan_dir)sS%(scan_number)05d/', 'dfile': None, 'distance': 7.19, 'energy': None, 'exp_string': 'exp_time', 'experimentID': None, 'hdr_thresholds': [500, 50000], 'label': None, 'lam': 6.5e-07, 'load_parallel': 'data', 'log_file_pattern': '%(base_path)sspec/dat-files/spec_started_2013_11_21_1659.dat', 'min_frames': 1, 'name': 'FliSpecScanMultexp', 'num_frames': None, 'orientation': (True, True, False), 'positions_theory': None, 'psize': 0.000172, 'psize_det': 2.4e-05, 'rebin': None, 'save': None, 'scan_dir': 'ccdfli/S00000-00999/', 'scan_number': 74, 'shape': 256, 'version': 0.1, 'z': 0.158}
check(frames_requested, start=0)

Override in subclass for custom implementation

This method checks how many frames the preparation routine may process, starting from frame start at a request of frames.

This method is supposed to return the number of accessible frames for preparation and should determine if data acquisition for this scan is finished. Its main purpose is to allow for a data acquisition scheme, where the number of frames is not known when PtyScan is constructed, i.e. a data stream or an on-the-fly reconstructions.

Note

If num_frames is set on __init__() of the subclass, this method can be left as it is.

Parameters

  • frames (int or None) – Number of frames requested.

  • start (int or None) – Scanpoint index to start checking from.

Returns

  • frames_accessible (int) – Number of frames readable.

  • end_of_scan (int or None) – is one of the following, - 0, end of the scan is not reached - 1, end of scan will be reached or is - None, can’t say

correct(raw, weights, common)

Override in subclass for custom implementation

Place holder for dark and flatfield correction. If load already provides data in the form of photon counts, and no frame specific weight is needed, this method may be left as it is.

May get merged with load in future.

Returns

data, weights – Flat and dark-corrected data dictionaries. These dictionaries must have the same keys as the input raw and contain corrected frames (data) and statistical weights (weights) which are zero for invalid or masked pixel other the number of detector counts that correspond to one photon count

Return type

dict

load(indices)

Override in subclass for custom implementation

Loads data according to node specific scanpoint indices that have been determined by LoadManager or otherwise.

Returns

raw, positions, weight – Dictionaries whose keys are the given scan point indices and whose values are the respective frame / position according to the scan point index. weight and positions may be empty

Return type

dict

Note

This is the most important method to change when subclassing PtyScan. Most often it suffices to override the constructor and this method to create a subclass suited for a specific experiment.

load_common()

Override in subclass for custom implementation

Called in initialize()

Loads anything and stores that in a dict. This dict will be available to all processes after initialize() through the attribute common

The purpose of this method is the same as load_weight() and load_positions() except for that the contents of common have no built-in effect of the behavior in the processing other than the user specifies it in py:meth:load

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

common

Return type

dict

ptypy.experiment.plugin module

Scan loading plugin. Meant to make easier user-generated, problem-specific data preparation.

This file is part of the PTYPY package.

copyright

Copyright 2014 by the PTYPY team, see AUTHORS.

license

see LICENSE for details.

ptypy.experiment.plugin.makeScanPlugin(pars=None)

Factory wrapper that provides a PtyScan object.

ptypy.experiment.spec module

Utility module to read spec files Adapted from spec_read.m by Andreas Menzel (PSI)

This file is part of the PTYPY package.

copyright

Copyright 2014 by the PTYPY team, see AUTHORS.

license

see LICENSE for details.

class ptypy.experiment.spec.SpecInfo(spec_filename)

Bases: object

parse(rehash=False)

Parse the spec dat-file and extract information.

class ptypy.experiment.spec.SpecScan

Bases: object

ptypy.experiment.spec.verbose(n, s)

This function should be replaced by the real verbose class after import. It is here for convenience since this module has no other external dependencies.

Module contents

Beamline-specific data preparation modules.

Currently available:

  • cSAXS

  • I13DLS, FFP and NFP

  • I08DLS, FFP and NFP

  • ID16A ESRF, NFP

  • AMO LCLS

  • DiProI FERMI

  • NanoMAX

This file is part of the PTYPY package.

copyright

Copyright 2014 by the PTYPY team, see AUTHORS.

license

see LICENSE for details.

class ptypy.experiment.FliSpecScanMultexp(pars=None, **kwargs)

Bases: ptypy.core.data.PtyScan

Class creation with minimum set of parameters, see PtyScan.DEFAULT Please note that class creation is not meant to load data.

Call initialize to begin loading and data file creation.

DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'base_path': '/data/CDI/opticslab_sxdm_2013/', 'center': 'auto', 'chunk_format': '.chunk%02d', 'dark_dir_pattern': '%(base_path)s%(scan_dir)sS%(dark_number)05d/', 'dark_number': 72, 'data_dir_pattern': '%(base_path)s%(scan_dir)sS%(scan_number)05d/', 'dfile': None, 'distance': 7.19, 'energy': None, 'exp_string': 'exp_time', 'experimentID': None, 'hdr_thresholds': [500, 50000], 'label': None, 'lam': 6.5e-07, 'load_parallel': 'data', 'log_file_pattern': '%(base_path)sspec/dat-files/spec_started_2013_11_21_1659.dat', 'min_frames': 1, 'name': 'FliSpecScanMultexp', 'num_frames': None, 'orientation': (True, True, False), 'positions_theory': None, 'psize': 0.000172, 'psize_det': 2.4e-05, 'rebin': None, 'save': None, 'scan_dir': 'ccdfli/S00000-00999/', 'scan_number': 74, 'shape': 256, 'version': 0.1, 'z': 0.158}
check(frames_requested, start=0)

Override in subclass for custom implementation

This method checks how many frames the preparation routine may process, starting from frame start at a request of frames.

This method is supposed to return the number of accessible frames for preparation and should determine if data acquisition for this scan is finished. Its main purpose is to allow for a data acquisition scheme, where the number of frames is not known when PtyScan is constructed, i.e. a data stream or an on-the-fly reconstructions.

Note

If num_frames is set on __init__() of the subclass, this method can be left as it is.

Parameters

  • frames (int or None) – Number of frames requested.

  • start (int or None) – Scanpoint index to start checking from.

Returns

  • frames_accessible (int) – Number of frames readable.

  • end_of_scan (int or None) – is one of the following, - 0, end of the scan is not reached - 1, end of scan will be reached or is - None, can’t say

correct(raw, weights, common)

Override in subclass for custom implementation

Place holder for dark and flatfield correction. If load already provides data in the form of photon counts, and no frame specific weight is needed, this method may be left as it is.

May get merged with load in future.

Returns

data, weights – Flat and dark-corrected data dictionaries. These dictionaries must have the same keys as the input raw and contain corrected frames (data) and statistical weights (weights) which are zero for invalid or masked pixel other the number of detector counts that correspond to one photon count

Return type

dict

load(indices)

Override in subclass for custom implementation

Loads data according to node specific scanpoint indices that have been determined by LoadManager or otherwise.

Returns

raw, positions, weight – Dictionaries whose keys are the given scan point indices and whose values are the respective frame / position according to the scan point index. weight and positions may be empty

Return type

dict

Note

This is the most important method to change when subclassing PtyScan. Most often it suffices to override the constructor and this method to create a subclass suited for a specific experiment.

load_common()

Override in subclass for custom implementation

Called in initialize()

Loads anything and stores that in a dict. This dict will be available to all processes after initialize() through the attribute common

The purpose of this method is the same as load_weight() and load_positions() except for that the contents of common have no built-in effect of the behavior in the processing other than the user specifies it in py:meth:load

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

common

Return type

dict

class ptypy.experiment.Hdf5Loader(pars=None, swmr=False, **kwargs)

Bases: ptypy.core.data.PtyScan

First attempt to make a generalised hdf5 loader for data. Please raise a ticket in github if changes are required so we can coordinate. There will be a Nexus and CXI subclass to this in the future.

hdf5 data loader

DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'darkfield': {'file': None, 'key': None}, 'dfile': None, 'distance': 7.19, 'electron_data': False, 'energy': 7.2, 'experimentID': None, 'flatfield': {'file': None, 'key': None}, 'framefilter': {'file': None, 'key': None}, 'frameorder': {'indices': None}, 'intensities': {'file': None, 'key': None}, 'label': None, 'load_parallel': 'data', 'mask': {'file': None, 'invert': False, 'key': None}, 'min_frames': 1, 'name': 'Hdf5Loader', 'normalisation': {'file': None, 'key': None, 'sigma': 3}, 'num_frames': None, 'orientation': None, 'outer_index': None, 'padding': None, 'positions': {'bounding_box': {'fast_axis_bounds': None, 'slow_axis_bounds': None}, 'fast_index': None, 'fast_key': None, 'fast_multiplier': 1.0, 'file': None, 'skip': 1, 'slow_index': None, 'slow_key': None, 'slow_multiplier': 1.0}, 'positions_theory': None, 'psize': 0.000172, 'rebin': None, 'recorded_distance': {'file': None, 'key': None, 'multiplier': 1.0}, 'recorded_energy': {'file': None, 'key': None, 'multiplier': 1.0, 'offset': 0.0}, 'recorded_psize': {'file': None, 'key': None, 'multiplier': 1.0}, 'save': None, 'shape': None, 'version': 0.1}
compute_scan_mapping_and_trajectory(data_shape, positions_fast_shape, positions_slow_shape)

This horrendous block of logic is all to do with making a semi-intelligent guess at what the data looks like.

get_corrected_intensities(index)

Corrects the intensities for darkfield, flatfield and normalisations if they exist. There is a lot of logic here, I wonder if there is a better way to get rid of it. Limited a bit by the MPI, and thinking about extension to large data size.

load_mapped_and_arbitrary_scan(indices)
load_mapped_and_raster_scan(indices)
load_unmapped_raster_scan(indices)
subtract_dark(raw, dark)

Subtract dark current from a raw frame and truncate negative values

class ptypy.experiment.Hdf5LoaderFast(pars=None, **kwargs)

Bases: ptypy.experiment.hdf5_loader.Hdf5Loader

hdf5 data loader

DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'darkfield': {'file': None, 'key': None}, 'dfile': None, 'distance': 7.19, 'electron_data': False, 'energy': 7.2, 'experimentID': None, 'flatfield': {'file': None, 'key': None}, 'framefilter': {'file': None, 'key': None}, 'frameorder': {'indices': None}, 'intensities': {'file': None, 'key': None}, 'label': None, 'load_parallel': 'data', 'mask': {'file': None, 'invert': False, 'key': None}, 'min_frames': 1, 'name': 'Hdf5Loader', 'normalisation': {'file': None, 'key': None, 'sigma': 3}, 'num_frames': None, 'orientation': None, 'outer_index': None, 'padding': None, 'positions': {'bounding_box': {'fast_axis_bounds': None, 'slow_axis_bounds': None}, 'fast_index': None, 'fast_key': None, 'fast_multiplier': 1.0, 'file': None, 'skip': 1, 'slow_index': None, 'slow_key': None, 'slow_multiplier': 1.0}, 'positions_theory': None, 'psize': 0.000172, 'rebin': None, 'recorded_distance': {'file': None, 'key': None, 'multiplier': 1.0}, 'recorded_energy': {'file': None, 'key': None, 'multiplier': 1.0, 'offset': 0.0}, 'recorded_psize': {'file': None, 'key': None, 'multiplier': 1.0}, 'save': None, 'shape': None, 'version': 0.1}
get_corrected_intensities(weights, intensities, index, indexed_frame_slice)

Corrects the intensities for normalisation and padding

load_mapped_and_arbitrary_scan(indices)
load_mapped_and_raster_scan(indices)
load_multiprocessing(src_slices)
load_unmapped_raster_scan(indices)
static subtract_dark(raw, dark)

Subtract dark current from a raw frame and truncate negative values

class ptypy.experiment.ID16AScan(pars=None, **kwargs)

Bases: ptypy.core.data.PtyScan

Subclass of PtyScan for ID16A beamline (specifically for near-field ptychography).

Default data parameters. See scan.data

Create a PtyScan object that will load ID16A data.

DEFAULT = {'add_poisson_noise': False, 'auto_center': False, 'base_path': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'dark_subtraction': False, 'det_flat_field': None, 'dfile': None, 'distance': 7.19, 'distortion_h_file': '/data/id16a/inhouse1/instrument/img1/optique_peter_distortion/detector_distortion2d_v.edf', 'distortion_v_file': '/data/id16a/inhouse1/instrument/img1/optique_peter_distortion/detector_distortion2d_h.edf', 'energy': 7.2, 'experimentID': None, 'flat_division': False, 'label': None, 'load_parallel': 'data', 'mask_file': None, 'min_frames': 1, 'motors': ['spy', 'spz'], 'motors_multiplier': 1e-06, 'name': 'ID16AScan', 'num_frames': None, 'orientation': (False, True, False), 'positions_theory': None, 'psize': 0.000172, 'rebin': None, 'recipe': {}, 'sample_name': None, 'save': None, 'shape': 256, 'version': 0.1, 'whitefield_file': '/data/id16a/inhouse1/instrument/whitefield/white.edf'}
check(frames, start=0)

This method checks how many frames the preparation routine may process, starting from frame start at a request of frames. Returns the number of frames available from starting index start, and whether the end of the scan was reached.

Parameters

  • frames (int or None) – Number of frames requested

  • start (int or None) – Scanpoint index to start checking from

Returns

  • frame_accessible (int) – Number of frames readable from a starting point start

  • end_of_scan (bool or None) – Check if the end of scan was reached, otherwise None if this routine doesn’t know

correct(raw, weights, common)

Place holder for dark and flatfield correction. Apply (eventual) corrections to the raw frames. Convert from “raw” frames to usable data.

Parameters

  • raw (dict) – Dict containing index matched data frames (np.array).

  • weights (dict) – Dict containing possible weights.

  • common (dict) – Dict containing possible dark and flat frames.

Returns

data, weights – Flat and dark-corrected data dictionaries. These dictionaries must have the same keys as the input raw and contain corrected frames (data) and statistical weights (weights)

Return type

dict

Note

If the negative values results from the calculation, they will be forced to be equal to 0.

load(indices)

Loads data according to node specific scanpoint indices that have been determined by LoadManager or otherwise.

Returns

raw, pos, weight – Dictionaries whose keys are the given scan point indices and whose values are the respective frame / position according to the scan point index. weight and positions may be empty

Return type

dict

Note

If one weight (mask) is to be used for the whole scan, it should be loaded with load_weights(). The same goes for the positions, which sould be loade with load_positions().

load_common()

Loads anything that is common to all frames and stores it in dict.

Returns

common – contains information common to all frames such as dark, flat-field, detector flat-field, normalization couter, and distortion files

Return type

dict

load_positions()

Loads all positions for all diffraction patterns in this scan.

Returns

Positions – A (N,2)-array where N is the number of positions.

Return type

ndarray

load_weight()

Function description see parent class. For now, this function will be used to load the mask.

Returns

weight2d – A two-dimensional array with a shape compatible to the raw diffraction data frames if provided from file

Return type

ndarray

class ptypy.experiment.MoonFlowerScan(pars=None, **kwargs)

Bases: ptypy.core.data.PtyScan

Test PtyScan class producing a romantic ptychographic data set of a moon illuminating flowers.

Override parent class default:

Parent pars are for the

DEFAULT = {'add_poisson_noise': True, 'auto_center': None, 'block_wait_count': 0, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'density': 0.2, 'dfile': None, 'distance': 7.19, 'energy': 7.2, 'experimentID': None, 'label': None, 'load_parallel': 'data', 'min_frames': 1, 'model': 'round', 'name': 'MoonFlowerScan', 'num_frames': 100, 'orientation': None, 'photons': 100000000.0, 'positions_theory': None, 'psf': 0.0, 'psize': 0.000172, 'rebin': None, 'save': None, 'shape': 128, 'version': 0.1}
check(frames=None, start=None)

Override in subclass for custom implementation

This method checks how many frames the preparation routine may process, starting from frame start at a request of frames.

This method is supposed to return the number of accessible frames for preparation and should determine if data acquisition for this scan is finished. Its main purpose is to allow for a data acquisition scheme, where the number of frames is not known when PtyScan is constructed, i.e. a data stream or an on-the-fly reconstructions.

Note

If num_frames is set on __init__() of the subclass, this method can be left as it is.

Parameters

  • frames (int or None) – Number of frames requested.

  • start (int or None) – Scanpoint index to start checking from.

Returns

  • frames_accessible (int) – Number of frames readable.

  • end_of_scan (int or None) – is one of the following, - 0, end of the scan is not reached - 1, end of scan will be reached or is - None, can’t say

load(indices)

Override in subclass for custom implementation

Loads data according to node specific scanpoint indices that have been determined by LoadManager or otherwise.

Returns

raw, positions, weight – Dictionaries whose keys are the given scan point indices and whose values are the respective frame / position according to the scan point index. weight and positions may be empty

Return type

dict

Note

This is the most important method to change when subclassing PtyScan. Most often it suffices to override the constructor and this method to create a subclass suited for a specific experiment.

load_positions()

Override in subclass for custom implementation

Called in initialize()

Loads all positions for all diffraction patterns in this scan. The positions loaded here will be available by all processes through the attribute self.positions. If you specify position on a per frame basis in load() , this function has no effect.

If theoretical positions positions_theory are provided in the initial parameter set DEFAULT, specifying positions here has NO effect and will be ignored.

The purpose of this function is to avoid reloading and parallel reads on files that may require intense parsing to retrieve the information, e.g. long SPEC log files. If parallel reads or log file parsing for each set of frames is not a time critical issue of the subclass, reimplementing this function can be ignored and it is recommended to only reimplement the load() method.

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

positions – A (N,2)-array where N is the number of positions.

Return type

ndarray

Note

Be aware that this method sets attribute num_frames in the following manner.

  • If num_frames == None : num_frames = N.

  • If num_frames < N , no effect.

  • If num_frames > N : num_frames = N.

load_weight()

Override in subclass for custom implementation

Called in initialize()

Loads a common (2d)-weight for all diffraction patterns. The weight loaded here will be available by all processes through the attribute self.weight2d. If a per-frame-weight is specified in load() , this function has no effect.

The purpose of this function is to avoid reloading and parallel reads. If that is not critical to the implementation, reimplementing this function in a subclass can be ignored.

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

weight2d – A two-dimensional array with a shape compatible to the raw diffraction data frames

Return type

ndarray

Note

For now, weights will be converted to a mask, mask = weight2d > 0 for use in reconstruction algorithms. It is planned to use a general weight instead of a mask in future releases.

class ptypy.experiment.PtyScan(pars=None, **kwargs)

Bases: object

PtyScan: A single ptychography scan, created on the fly or read from file.

BASECLASS

Objectives:

  • Stand alone functionality

  • Can produce .ptyd data formats

  • Child instances should be able to prepare from raw data

  • On-the-fly support in form of chunked data.

  • mpi capable, child classes should not worry about mpi

Default data parameters. See scan.data

Class creation with minimum set of parameters, see PtyScan.DEFAULT Please note that class creation is not meant to load data.

Call initialize to begin loading and data file creation.

CODES = {'msg1': 'Scan unfinished. More frames available after a pause', 'msgEOS': 'End of scan reached'}
DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'dfile': None, 'distance': 7.19, 'energy': 7.2, 'experimentID': None, 'label': None, 'load_parallel': 'data', 'min_frames': 1, 'name': 'PtyScan', 'num_frames': None, 'orientation': None, 'positions_theory': None, 'psize': 0.000172, 'rebin': None, 'save': None, 'shape': 256, 'version': 0.1}
EOS = 'msgEOS'
METAKEYS = ['version', 'num_frames', 'label', 'shape', 'psize', 'energy', 'center', 'distance']

Keys to store in meta param

WAIT = 'msg1'
property abort
auto(frames)

Repeated calls to this function will process the data.

Parameters

frames (int) – Number of frames to process.

Returns

one of the following

  • WAIT, if scan’s end is not reached, but no data could be prepared yet

  • EOS, if scan’s end is reached

  • a data package otherwise

Return type

variable

check(frames=None, start=None)

Override in subclass for custom implementation

This method checks how many frames the preparation routine may process, starting from frame start at a request of frames.

This method is supposed to return the number of accessible frames for preparation and should determine if data acquisition for this scan is finished. Its main purpose is to allow for a data acquisition scheme, where the number of frames is not known when PtyScan is constructed, i.e. a data stream or an on-the-fly reconstructions.

Note

If num_frames is set on __init__() of the subclass, this method can be left as it is.

Parameters

  • frames (int or None) – Number of frames requested.

  • start (int or None) – Scanpoint index to start checking from.

Returns

  • frames_accessible (int) – Number of frames readable.

  • end_of_scan (int or None) – is one of the following, - 0, end of the scan is not reached - 1, end of scan will be reached or is - None, can’t say

correct(raw, weights, common)

Override in subclass for custom implementation

Place holder for dark and flatfield correction. If load already provides data in the form of photon counts, and no frame specific weight is needed, this method may be left as it is.

May get merged with load in future.

Returns

data, weights – Flat and dark-corrected data dictionaries. These dictionaries must have the same keys as the input raw and contain corrected frames (data) and statistical weights (weights) which are zero for invalid or masked pixel other the number of detector counts that correspond to one photon count

Return type

dict

property end_of_scan
property frames_accessible
get_data_chunk(chunksize, start=None)

This function prepares a container that is compatible to data package.

This function is called from the auto() function.

info

Param container that stores all input parameters.

initialize()

Begins the Data preparation and intended as the first method that does read-write access on (large) data. Does the following:

load(indices)

Override in subclass for custom implementation

Loads data according to node specific scanpoint indices that have been determined by LoadManager or otherwise.

Returns

raw, positions, weight – Dictionaries whose keys are the given scan point indices and whose values are the respective frame / position according to the scan point index. weight and positions may be empty

Return type

dict

Note

This is the most important method to change when subclassing PtyScan. Most often it suffices to override the constructor and this method to create a subclass suited for a specific experiment.

load_common()

Override in subclass for custom implementation

Called in initialize()

Loads anything and stores that in a dict. This dict will be available to all processes after initialize() through the attribute common

The purpose of this method is the same as load_weight() and load_positions() except for that the contents of common have no built-in effect of the behavior in the processing other than the user specifies it in py:meth:load

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

common

Return type

dict

load_positions()

Override in subclass for custom implementation

Called in initialize()

Loads all positions for all diffraction patterns in this scan. The positions loaded here will be available by all processes through the attribute self.positions. If you specify position on a per frame basis in load() , this function has no effect.

If theoretical positions positions_theory are provided in the initial parameter set DEFAULT, specifying positions here has NO effect and will be ignored.

The purpose of this function is to avoid reloading and parallel reads on files that may require intense parsing to retrieve the information, e.g. long SPEC log files. If parallel reads or log file parsing for each set of frames is not a time critical issue of the subclass, reimplementing this function can be ignored and it is recommended to only reimplement the load() method.

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

positions – A (N,2)-array where N is the number of positions.

Return type

ndarray

Note

Be aware that this method sets attribute num_frames in the following manner.

  • If num_frames == None : num_frames = N.

  • If num_frames < N , no effect.

  • If num_frames > N : num_frames = N.

load_weight()

Override in subclass for custom implementation

Called in initialize()

Loads a common (2d)-weight for all diffraction patterns. The weight loaded here will be available by all processes through the attribute self.weight2d. If a per-frame-weight is specified in load() , this function has no effect.

The purpose of this function is to avoid reloading and parallel reads. If that is not critical to the implementation, reimplementing this function in a subclass can be ignored.

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

weight2d – A two-dimensional array with a shape compatible to the raw diffraction data frames

Return type

ndarray

Note

For now, weights will be converted to a mask, mask = weight2d > 0 for use in reconstruction algorithms. It is planned to use a general weight instead of a mask in future releases.

min_frames

Minimum number of frames to prepare / load with call of auto()

num_frames

Total number of frames to prepare / load. Set by num_frames

post_initialize()

Placeholder. Called at the end of initialize() by all processes.

Use this method to benefit from ‘hard-to-retrieve but now available’ information after initialize.

report(what=None, shout=True)

Make a report on internal structure.

class ptypy.experiment.PtydScan(pars=None, **kwargs)

Bases: ptypy.core.data.PtyScan

PtyScan provided by native “ptyd” file format.

PtyScan provided by native “ptyd” file format.

DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'dfile': None, 'distance': 7.19, 'energy': 7.2, 'experimentID': None, 'label': None, 'load_parallel': 'data', 'min_frames': 1, 'name': 'PtydScan', 'num_frames': None, 'orientation': None, 'positions_theory': None, 'psize': 0.000172, 'rebin': None, 'save': None, 'shape': 256, 'source': 'file', 'version': 0.1}
check(frames=None, start=None)

Implementation of the check routine for a .ptyd file format.

See also

PtyScan.check

load(indices)

Load from ptyd.

Due to possible chunked data, slicing frames is non-trivial.

load_positions()

Override in subclass for custom implementation

Called in initialize()

Loads all positions for all diffraction patterns in this scan. The positions loaded here will be available by all processes through the attribute self.positions. If you specify position on a per frame basis in load() , this function has no effect.

If theoretical positions positions_theory are provided in the initial parameter set DEFAULT, specifying positions here has NO effect and will be ignored.

The purpose of this function is to avoid reloading and parallel reads on files that may require intense parsing to retrieve the information, e.g. long SPEC log files. If parallel reads or log file parsing for each set of frames is not a time critical issue of the subclass, reimplementing this function can be ignored and it is recommended to only reimplement the load() method.

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

positions – A (N,2)-array where N is the number of positions.

Return type

ndarray

Note

Be aware that this method sets attribute num_frames in the following manner.

  • If num_frames == None : num_frames = N.

  • If num_frames < N , no effect.

  • If num_frames > N : num_frames = N.

load_weight()

Override in subclass for custom implementation

Called in initialize()

Loads a common (2d)-weight for all diffraction patterns. The weight loaded here will be available by all processes through the attribute self.weight2d. If a per-frame-weight is specified in load() , this function has no effect.

The purpose of this function is to avoid reloading and parallel reads. If that is not critical to the implementation, reimplementing this function in a subclass can be ignored.

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

weight2d – A two-dimensional array with a shape compatible to the raw diffraction data frames

Return type

ndarray

Note

For now, weights will be converted to a mask, mask = weight2d > 0 for use in reconstruction algorithms. It is planned to use a general weight instead of a mask in future releases.

class ptypy.experiment.QuickScan(pars=None, **kwargs)

Bases: ptypy.core.data.PtyScan

Test PtyScan to benchmark graph creation further down the line.

Override parent class default:

Parent pars are for the

DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'density': 0.05, 'dfile': None, 'distance': 7.19, 'energy': 7.2, 'experimentID': None, 'label': None, 'load_parallel': 'data', 'min_frames': 1, 'name': 'MoonFlowerScan', 'num_frames': 100, 'orientation': None, 'positions_theory': None, 'psize': 0.000172, 'rebin': None, 'save': None, 'shape': 64, 'version': 0.1}
load(indices)

Override in subclass for custom implementation

Loads data according to node specific scanpoint indices that have been determined by LoadManager or otherwise.

Returns

raw, positions, weight – Dictionaries whose keys are the given scan point indices and whose values are the respective frame / position according to the scan point index. weight and positions may be empty

Return type

dict

Note

This is the most important method to change when subclassing PtyScan. Most often it suffices to override the constructor and this method to create a subclass suited for a specific experiment.

load_positions()

Override in subclass for custom implementation

Called in initialize()

Loads all positions for all diffraction patterns in this scan. The positions loaded here will be available by all processes through the attribute self.positions. If you specify position on a per frame basis in load() , this function has no effect.

If theoretical positions positions_theory are provided in the initial parameter set DEFAULT, specifying positions here has NO effect and will be ignored.

The purpose of this function is to avoid reloading and parallel reads on files that may require intense parsing to retrieve the information, e.g. long SPEC log files. If parallel reads or log file parsing for each set of frames is not a time critical issue of the subclass, reimplementing this function can be ignored and it is recommended to only reimplement the load() method.

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

positions – A (N,2)-array where N is the number of positions.

Return type

ndarray

Note

Be aware that this method sets attribute num_frames in the following manner.

  • If num_frames == None : num_frames = N.

  • If num_frames < N , no effect.

  • If num_frames > N : num_frames = N.

load_weight()

Override in subclass for custom implementation

Called in initialize()

Loads a common (2d)-weight for all diffraction patterns. The weight loaded here will be available by all processes through the attribute self.weight2d. If a per-frame-weight is specified in load() , this function has no effect.

The purpose of this function is to avoid reloading and parallel reads. If that is not critical to the implementation, reimplementing this function in a subclass can be ignored.

If load_parallel is set to all or common`, this function is executed by all nodes, otherwise the master node executes this function and broadcasts the results to other nodes.

Returns

weight2d – A two-dimensional array with a shape compatible to the raw diffraction data frames

Return type

ndarray

Note

For now, weights will be converted to a mask, mask = weight2d > 0 for use in reconstruction algorithms. It is planned to use a general weight instead of a mask in future releases.

class ptypy.experiment.SimScan(pars=None, **kwargs)

Bases: ptypy.core.data.PtyScan

Simulates a ptychographic scan and acts as Virtual data source.

Parameters

pars (Param) – PtyScan parameters. See scandata.SimScan.

DEFAULT = {'add_poisson_noise': False, 'auto_center': None, 'center': 'fftshift', 'chunk_format': '.chunk%02d', 'detector': {}, 'dfile': None, 'distance': 7.19, 'energy': 7.2, 'experimentID': None, 'frame_size': None, 'illumination': {'aperture': {'central_stop': None, 'diffuser': None, 'edge': 2.0, 'form': 'circ', 'offset': 0.0, 'rotate': 0.0, 'size': None}, 'diversity': {'noise': (0.5, 1.0), 'power': 0.1, 'shift': None}, 'model': None, 'photons': None, 'propagation': {'antialiasing': 1, 'focussed': None, 'parallel': None, 'spot_size': None}, 'recon': {'label': None, 'rfile': '\\*.ptyr'}}, 'label': None, 'load_parallel': 'data', 'min_frames': 1, 'name': 'SimScan', 'num_frames': None, 'orientation': None, 'plot': True, 'pos_drift': 0.0, 'pos_noise': 1e-10, 'pos_scale': 0.0, 'positions_theory': None, 'propagation': 'farfield', 'psf': None, 'psize': 0.000172, 'rebin': None, 'sample': {'diversity': {'noise': None, 'power': 0.1, 'shift': None}, 'fill': 1, 'model': None, 'process': {'density': 1, 'formula': None, 'offset': (0, 0), 'ref_index': (0.5, 0.0), 'smoothing': 2, 'thickness': 1e-06, 'zoom': None}, 'recon': {'rfile': '\\*.ptyr'}, 'stxm': {'label': None}}, 'save': None, 'shape': 256, 'verbose_level': 1, 'version': 0.1, 'xy': {'count': None, 'extent': 1.5e-05, 'jitter': None, 'model': None, 'offset': 0.0, 'override': None, 'spacing': 1.5e-06, 'steps': 10}}
load(indices)

Load data, weights and positions from internal dictionarys

manipulate_ptycho(ptycho)

Overwrite in child class for inline manipulation of the ptycho instance that is created by the Simulation