Table of supported image formats in EMAN2

File format conversions

• Most EMAN2 programs will read and write any of the formats above directly without conversion.

• e2proc2d.py and e2proc3d.py provide options for saving in specific formats with specific data modes.

• The Save as button in the e2display.py and e2projectmanager.py browsers can be used to save into an arbitrary format.

Special issues for MRC/MRCS/CCP4 files

MRC/CCP4 format supports a single 1-D, 2-D or 3-D image, with an associated header. At some point in time, someone decided it would be a good idea to store sets of 2-D particle images as "stacks" in 3-D. That is, a set of NZ identically sized NX x NY images are stacked to make a single 3-D pseudo-volume image. The problem is that the original format was not designed for this, and historically there was no consistent way a program could tell if an MRC file contains a true volume (like a 3-D reconstruction) or a stack of 2-D images. While a number of developers have recently agreed upon a standard way of doing this in future, the last 30 years of files floating around in the community don't have this information stored in a consistent way. As of EMAN2.1, stack files should use the ".mrcs" extension and single volumes should use the ".mrc" extension. ".mrc" files will always be read as if they contain a single image, and ".mrcs" files can never be 3-D. This may evolve in the future as the new standards become more refined.

Additionally, there are options in the e2proc2d.py command which will treat single volume files as stacks of images without the .mrcs extension (again, if you just use the .mrcs extension, these methods should not be required) :

• --threed2twod - reads an MRC-style stack file and outputs to a 'normal' set of 2-D images (used by all other file formats supporting multiple images)

• --twod2threed - reads a set of 2-D images and outputs an MRC-style stack

• --threed2threed - reads an MRC-style stack file and outputs to another MRC-style stack file

These options can also be used with other file formats.

Special issues for EER files

EER is a specialized format for the Falcon4 camera which records actual pixel events at a very high effective framerate. Individual frames are RLE encoded, so despite storing up to 4x superresolution counting mode images, file size is still smaller than a gain corrected MRC stack averaged to 30 FPS.

The default reader will operate without oversampling. You will need to specify a specific filetype with e2proc2d.py to read super-resolution data instead (8k x 8k or 16k x 16k).

To make use of these files you normally also need to have the appropriate gain reference image from the Falcon 4, which at the time of this writing is stored in FEIRAW format. There is a program in examples, which can convert FEIRAW files to any other format you like, but they aren't natively read by other EMAN2 programs.

Several bugs were fixed in mid-October 2020, so it is critical that you use a version dated 10/22/20 or later!

Here is a workflow for processing gain references with EER files:

   1 examples/feiraw2hdf.py gain_post_ec_eer.raw gain_post_ec_eer.hdf
   2 e2proc2d.py gain_post_ec_eer.hdf gain_norm.hdf --process math.reciprocal
   3 e2proc2d.py myimage.eer moviestack.mrcs --avgseq 60
   4 e2proc2d.py moviestack.mrcs moviestack.mrcs --inplace --mult gain_norm.hdf

• It may be possible to merge steps 3 and 4.
• there was previously a --translate 1,0 in step 2, but that should no longer be necessary. If you observe a problem which seems to be due to the gain image being translationally misaligned, please report it on the Google group

• The --avgseq 60 options says to average sets of 60 frames in the original EER file, so an 1800 frame EER would produce a 30 frame normalized movie. 60 can be changed to any desired value.

Reading and Writing images in Python (for programmers)

The main image object in EMAN2 is called EMData(). EMData objects represent an image in an arbitrary file format in the computer's memory, with arbitrary associated tag-based metadata.

Simple Image Reading/Writing

The specification for reading/writing images is:

   1 # note that optional arguments [ ] below require all previous arguments to be specified
   2 
   3 # Read multiple images at once, class-method
   4 imagelist=EMData.read_images(filename,[image#_list],[header_only])
   5 
   6 # Read a single image
   7 img=EMData()
   8 img.read_image(filename,[image#],[header_only],[Region])
   9 # or
  10 img=EMData(filename,[image#],[header_only])
  11 
  12 
  13 # write a single image
  14 EMData.write_image(filename,image#,[filetype],[header_only],[Region],[Datatype])

Where filename, is the name of the file containing the image data, in any supported format, image# is the zero-indexed image number within the file, image#_list is a python list or tuple of image numbers, header_only is a boolean flag indicating that only the header should be read/written from/to the file, Region is a Region(x0,y0,xsize,ysize) or Region(x0,y0,z0,xsize,ysize,zsize) object.

Filetype can be : IMAGE_UNKNOWN, IMAGE_AMIRA, IMAGE_IMAGIC, IMAGE_PIF, IMAGE_SPIDER, IMAGE_VTK, IMAGE_DM3, IMAGE_GATAN2, IMAGE_LST, IMAGE_PNG, IMAGE_TIFF, IMAGE_XPLOR, IMAGE_DM4, IMAGE_HDF, IMAGE_MRC, IMAGE_SAL, IMAGE_EM, IMAGE_ICOS, IMAGE_PGM, IMAGE_SINGLE_SPIDER, IMAGE_V4L. If IMAGE_UNKNOWN is used on write, then the file extension will be used to determine the filetype. Note that since MRC format does not distinguish between 3-D volumes and stacks of 2-D images, the '.mrcs' extension MUST be used for stack files, and the '.mrc' extension MUST be used for non-stack volume data.

Datatype can be: EM_CHAR, EM_FLOAT, EM_INT, EM_UINT, EM_USHORT, EM_DOUBLE, EM_FLOAT_COMPLEX, EM_SHORT, EM_UCHAR, EM_USHORT_COMPLEX, EM_SHORT_COMPLEX. While SHORT_COMPLEX types are defined, they should never be actually used. FLOAT_COMPLEX is only really usable for HDF files. Strongly suggest not reading/writing complex images, and simply recomputing the FFT instead. Not all file formats support all data types!

If no image#_list is specified to read_images, then ALL images in the file will be read in.

   1 # Create a new EMData object and initializes it with the first image in "myimage.hdf".
   2 # This will work with any supported file format, not just HDF
   3 img=EMData("myimage.hdf")
   4 
   5 # Replace the data in EMdata object 'img' with the 3rd image from "myimage.hdf" (the first is #0)
   6 img.read_image("myimage.hdf",2)
   7 
   8 # Write an EMData object to disk as the 3rd image in "image.hdf"
   9 img.write_image("image.hdf",2)
  10 
  11 # Read all of the images from the SPIDER stack file (also works with single image files) "test.spi"
  12 # lst will become a list of EMData objects
  13 lst=EMData.read_images("test.spi")
  14 
  15 # Count the number of images available in a stack file
  16 n=EMUtil.get_image_count("myimage.hdf")
  17 
  18 # Create a new EMData object with ONLY HEADER INFORMATION from the 5th image
  19 # in the "myimage.hdf" stack file. Any image processing operations on this object
  20 # will cause EMAN2 to crash, because it doesn't have data loaded for the actual image.
  21 # This can be useful when all you need is the header information from a bunch of images.
  22 hdr=EMData("myimage.hdf",4,True)

Region I/O

Region I/O permits reading or writing sub-images/volumes from within a file. It is not supported for all file formats. This is useful when processing huge files (like full 4k tomograms) on machines with limited RAM. For region reading, it is possible to specify a Region extending outside the actual image dimensions, though this generally isn't a good idea. For region writing, the region must be completely inside image bounds.

   1 # Read a subimage with origin (1,1,1) and size 8x8x8
   2 img = EMData
   3 region = Region(1,1,1,8,8,8)
   4 img.read_image("3dimage.hdf",0,False,region)

Storage type

Internally EMAN2 stores all images as 32-bit (single precision) floating point. Many file formats also support other storage modes. The various formats are defined in a dictionary imported from EMAN2.py: file_mode_map. There is also a file_mode_range dictionary which contains the numeric limits for each type. If you set the header values renfer_min and render_max in each image before writing, this will control how the float data is scaled to the specified mode. ie - if render_min is 0 and render_max is 1.0, then the 0-1 range in the internal image will be mapped to the full available scale of (integer mode) output formats. Note also that not all file formats support all modes.

Here are some examples of how to write in alternative formats:

img = EMData(128,128)
img.write('float-image.mrc')  #by default, image will be write as float
img.write_image('short-image.mrc', 0, IMAGE_MRC, False, None, EM_SHORT) #write mrc file in short (16bit)
img.write_image('byte-image.mrc', 0, IMAGE_MRC, False, None, EM_UCHAR) #write mrc file in byte (8bit)
img.write_image('byte-image.spi', 0, IMAGE_UNKNOWN, False, None, EM_FLOAT) #write mrc file in byte (8bit)