EMAN2 Output Files

This page documents all of the various files produced by various tasks and workflows in EMAN2. While the format of the actual files will be one of the standard EMAN2 supported image formats in most cases, these pages will explain the contents of files with specific standard names.

• File formats & data storage conventions
• Parameters/Metadata appearing in EMData attributes (and stored on disk with the image)

Older EMAN2 refinements made use of 2-D classification as part of the 3-D refinement process. Thanks to higher contrast data from direct detectors, the new refinement strategy determines the orientation of individual particles directly, so the results are much easier to interpret.

Each run will create a numbered r3d_XX folder. Input files:

• –ptcls specifies the input particles in .lst format. This input file will become ptcls_00.lst in the output folder
• –ref specifies the initial model to start the refinement. This will be phase randomized beyond the initial resolution two times to produce threed_00_even.hdf and threed_00_odd.hdf
• The full set of input parameters from the command-line (or assumed defaults) will be saved to 0_spa_params.json (https://blake.bcm.edu/emanwiki/Eman2JSStorage)

Output files (in temporal order of creation):

• ptcls_XX.lst - LST file containing the particles and their orientations in the current iteration. Functionally similar to a STAR file in Relion, but with other benefits, since LST files can be treated as images. The orientation metadata will be treated as part of the image header when read. See ImageFormats for details.
• threed_XX.hdf - final 3-D map for iteration XX
• threed_XX_even.hdf and threed_XX_odd.hdf - independent even/odd maps (“gold standard” refinement). These are masked and filtered.
• threed_even_unmasked.hdf and threed_odd_unmasked.hdf - unmasked/unfiltered even/odd maps for the last completed iteration. These are overwritten after each iteration completes
• fsc_masked_XX.txt - Fourier shell correlation curve between even/odd maps with a conservative soft mask applied. This can be used for a good, conservative resolution estimate.
• fsc_maskedtight_XX.txt - Similar, but with a more aggressive mask, similar to Relion post-processing. This is conventionally what people would use in publication.
• fsc_unmasked.txt_XX - Fourier shell correlation curve between even/odd maps with no applied mask. This is sometimes viewed as the most “conservative” resolution curve, however it is also susceptible to box size issues. ie - this curve will become worse if a larger box size is used, even though the structure will generally be better.
• fscvol_xx.hdf - only present if local filtration is used in refinement. This map estimates the local resolution at each point in space. It can be used in Chimera to color the surface of the threed_XX.hdf file to highlight regions with better/worse resolution.
• mask.hdf and masktight.hdf - mask files used in the last complete iteration of refinement

Input files:

• files specified via –input, –model for e2refine_easy
• files specified via –input and –model or –models
• strucfac.txt should normally be present in the project directory - This is a text file containing the ideal 1-D structure factor expected for the final map. Intensity as a function of spatial frequency.

Output files (in temporal order of creation), _xx denotes the iteration number:

• projections_xx.hdf, proj_stg1_xx.hdf - Map projection files
• simmx_xx.hdf, simmx_stg1_xx.hdf, proj_simmx_xx.hdf - Similarity matrix image files
• classmx_xx.hdf - Classification matrix image files
• cls_result_xx.hdf - Class-Averaging Results matrix image files
• classes_xx.hdf - Class-averages
• threed_even_unmasked.hdf, threed_odd_unmasked.hdf - Reconstructions from the final completed iteration without masking or final filtration, suitable for use with ResMap
• threed_xx.hdf, threed_filt_xx.hdf, threed_mask_xx.hdf - 3-D reconstructions

If you are struggling with a failed refinement, look at the produced files in this order until you find something unexpected, and that may give some clues as to what went wrong. Don't be shy about posting to the Google Group for help!

The program e2evalrefine.py –evalptclqual is normally used to assess the quality of individual particles after running e2refine_easy. This produces a file in the Project Folder called ptclfsc_XX.txt where XX is the number of the refine_XX folder. This file is a multicolumn text file with the following contents. It can be displayed with e2display –plot :

• Integrated particle/projection FSC from 100 - 30 A
  • For even marginally good data, this should always be positive, and have a pretty good spread between 0 and 1
• Integrated particle/projection FSC from 30 - 15 A
  • If there is any hope of subnanometer resolution, this should also be mostly positive and have a good correlation with the first column
• Integrated particle/projection FSC from 15 - 8 A
  • For _fullres or _lp5 data suitable for sub-5A resolution, you should see some correlation here with column 0, and the values should have a clear positive bias.
• Integrated particle/projection FSC from 8 - 4 A
  • Even for very good _fullres data, this column is rarely useful. Normally it will look like a symmetric distribution about 0
• “alt” Euler angle for this particle
• “az” Euler angle
• class number
• defocus
• particle number within file
• file the particle is from
• projection file number (optional)
• projection file name (optional)

You may also wish to look at: e2refine2d Input files:

• file specified via –input

Output files:

• input_fp - rotational/translational invariants for each particle
• input_fp_basis - MSA basis vectors (images) from input_fp
• input_fp_basis_proj - MSA subspace projections of the input_fp invariants
• classmx_00 - Initial classification of particles, same format as classmx above
• classes_init - Initial set of class-averages from invariant method (not very good usually)
• allrefs_XX - All of the references (sorted and aligned) to be used for the current iteration. Other than sorting/alingment, same as classes_XX files
• basis_XX - MSA basis from allrefs_xx
• aliref_XX - Subset of allrefs used for alignment of raw particles
• simmx_XX - Similarity matrix in same format as simmx above
• input_XX_proj - Aligned particles projected into basis_XX subspace
• classmx_XX - Classification matrix for the current iteration (as above)
• classes_XX - Class averages at the end of the iteration. The highest numbered classes_XX file is the final output of the program

Output Files:

• align_ref.hdf - contains 2 volumes representing the even and odd references respectively. If –goldstandard isn't specified, then the two references will be identical
• particle_parms.json - contains metadata for each subtomogram indexed by (filename,number) tuple
• aliptcls.hdf - Requires specifying –saveali. Stack of aligned particles. Note that particles are not split even/odd even if –goldstandard is used, which differs from e2spt_classaverage. These particles are for user-use only, they are not normally used in generating averages.

Input files:

• –input, subvolume stack in .hdf format
• –ref, if performing reference-based refinement, reference image in .hdf format

Output files:

• sptbt, spthac or sptssa subdirectories if –ref is not provided and the program generates initial models automatically. The corresponding directory will appear, with the files specified above in it, depending on whether –btref, –hacref or –ssaref are specified.

• aliptcls.hdf - Requires specifying –saveali. Stack of final aligned particles from last refinement iteration. If gold-standard refinement is on (by not supplying –goldstadardoff) the output includes aliptcls_even.hdf and aliptcls_odd.hdf.

• avgs.hdf - Requires specifying –savesteps. Stack of the averages produced in all iterations of refinement. If gold-standard refinement is on (by not supplying –goldstadardoff) avgs.hdf; will be the averages of the odd and even averages for each iteration, and additionalavgs_even.hdf and avgs_odd.hdf stacks will be generated containing the even and odd averages across iterations.

• classmx_XX.hdf - Classification matrix for the current iteration (as above)

• final_avg.hdf - Final average of the final even and odd averages (or simply the final average of all the particles if gold-standard refinement is off).

• fsc_XX.txt - FSC between even and odd averages for the current iteration.

• initialrefs_fsc.txt - Initial FSC curve against the reference or between the initial even and odd models generated by the program if not reference is provided via –ref.

• parameters_sptclassavg.txt - Text file containing the executed command and the values used by the program for all parameters (including defaulted ones, not specified by the user).

• spt_cccs_XX.txt - Text file with sorted cross correlation coefficients. If gold-standard refinement is on, spt_cccs_XX_even.txt and spt_cccs_XX_odd.txt are generated.

• spt_meanccc.txt - Text file containing the mean cross correlation coefficient across iterations (this can help determine whether the mean score is improving or has plateaued or is degenerating). If gold-standard refinement is on, spt_meanccc_even.txt and spt_meanccc_odd.txt are generated.

• tomo_xforms_0.json - Json file with alignment parameters for all particles in the stack.

• tomo_xforms_0_avgali2ref.json - Json file with alignment parameters for the final average aligned to the reference if gold-standard refinement is off. This becomes tomo_xforms_0_oddali2even.json if gold-standard refinement is on.