Input/Output Parameters

Input/Output Parameters#

Learning more about the io section of the parameter tree - how to save the final result and intermediates into .ptyr files, and how to use live plotting with Jupyter notebooks.

In the p.io section of the parameter tree we can specify where we would like our output to be written to. First, we can set a root directory for all further input/output operations, e.g. “/tmp”

p.io.home = "/tmp"

PtyPy saves its results into .ptyr files which are basically HDF5 files and can therefore be inspected by any software/library that supports the HDF5 format, e.g. h5py. We can now specify that the main output file should be placed in a subfolder recon relative to the root directory

p.io.rfile =  "recons/%(run)s_%(engine)s_%(iterations)04d.ptyr"

where %(run)s is dynamically replaced by the string defined in p.run, %(engine)s by the name of the engine used for the reconstructions and %(iterations)04d by the total number of iterations at the end formatted as a 4-digit integer.

Live-plotting#

And finally, PtyPy has some live-viewing capabilities that are controlled with p.io.interaction and p.io.autoplot. This feature has mostly been designed to run in a threaded mode where the plotting process is separate from the main processing thread. This server-client setup can also be used in situations where the plotting client is running on a different machine than the reconstruction (server), see Live visualisation for more details. However, since we are working with Jupyter notebooks we won’t need any threading and can do all the plotting inside the notebook using matlotlib’s inline functionality by setting the following parameters

p.io.interaction = u.Param(active=False)
p.io.autoplot = u.Param()
p.io.autoplot.active = True
p.io.autoplot.threaded = False
p.io.autoplot.layout = "jupyter"
p.io.autoplot.interval = 10

which will display the current status of the reconstruction after every block of iterations as specified by p.io.autoplot.interval.

Note

Live-plotting in Jupyter notebooks using p.io.autoplot works best in conjunction with p.verbose_level = “interactive”.

Saving intermediates to file#

There is also the option to auto-save intermediate states of the reconstruction at a given interval into a subfolder dumps relative to the root directory

p.io.autosave = u.Param()
p.io.autosave.active = True
p.io.autosave.interval = 10
p.io.autosave.rfile = 'dumps/%(run)s_%(engine)s_%(iterations)04d.ptyr'

where %(run)s, %(engine)s and %(iterations)04d are again dynamic labels.

Challenge

Modify the p.io.home path in the example below to wherever you would like the PtyPy output to be, run the reconstruction and observe all the new files saved to the path specified as p.io.home. You can use ptypy.utils.plot_recon_from_ptyr to look at the MoonFlower reconstruction at different iterations.

Tip

You can use shell commands within Jupyter Notebooks to explore the filesystem, i.e. ls /tmp to list all files in /tmp.

import ptypy
import ptypy.utils as u

p = u.Param()
p.verbose_level = "interactive"

# Run label (ID)
p.run = "moonflower"

# I/O settings
p.io = u.Param()

# Set the root path for all input/output operations
# Change this to wherever you would like 
p.io.home = "/tmp"

# Path to final .ptyr output file 
# using variables p.run, engine name and total nr. of iterations
p.io.rfile =  "recons/%(run)s_%(engine)s_%(iterations)04d.ptyr"

# Turn off interaction server
p.io.interaction = u.Param(active=False)

# Use non-threaded live plotting
p.io.autoplot = u.Param()
p.io.autoplot.active=True
p.io.autoplot.threaded = False
p.io.autoplot.layout = "jupyter"
p.io.autoplot.interval = 10

# Save intermediate .ptyr files (dumps) every 10 iterations
p.io.autosave = u.Param()
p.io.autosave.active = True
p.io.autosave.interval = 10
p.io.autosave.rfile = 'dumps/%(run)s_%(engine)s_%(iterations)04d.ptyr'

p.scans = u.Param()
p.scans.MF = u.Param()
p.scans.MF.name = "Full"
p.scans.MF.data= u.Param()
p.scans.MF.data.name = "MoonFlowerScan"
p.scans.MF.data.shape = 128
p.scans.MF.data.num_frames = 200
p.scans.MF.data.save = None
p.scans.MF.data.density = 0.2
p.scans.MF.data.photons = 1e8
p.scans.MF.data.psf = 0

p.engines = u.Param()
p.engines.engine00 = u.Param()
p.engines.engine00.name = "DM"
p.engines.engine00.numiter = 80
p.engines.engine00.numiter_contiguous = 5

P = ptypy.core.Ptycho(p,level=5)

import os
import ptypy.utils.plot_client as pc
ptyr_filename = os.path.join(p.io.home, "dumps/moonflower_DM_0010.ptyr")
fig = pc.figure_from_ptyr(ptyr_filename)
Contents