Porting an EMAN1 refinement project to EMAN2

Note : This page is for those who hate to read. The tutorials provide a much better way of learning EMAN2. This will get you started if you just want to rapidly switch a project from EMAN1, though. Just remember if you find yourself asking 'but what does THAT mean' when you read this, you're reading the wrong page :^)

A Quickstart Guide

This quickstart makes use of the workflow, as the goal here is presumably to get the best results in EMAN2, and that is most easily accomplished in the workflow. It will take you all the way through a canonical reconstruction starting with data you've already processed in EMAN1 (or any other image processing package for that matter). This is written targeting primarily Linux/Mac users, but hopefully windows users will be able to follow along.

For purposes of this tutorial will call the directory where your eman1 data resides 'eman1'. You should also make another directory where your EMAN2 project will reside. We will call this 'eman2', but you can use any name you like.

Initial Setup

1. It is very important that you cd eman2 before starting the EMAN2 GUI. Whatever directory you are in when you run e2workflow.py will become your project directory.

2. Run e2workflow.py. Two GUI windows should appear. One will show the status of running background jobs. The other will contain a collapsible tree of workflow items.

3. In the workflow window, click on the Single Particle Reconstruction item.

   • Note that you are not expanding the list here, but clicking on the actual words.
   • This should pop up a window containing project parameters.
   • Fill in the first 4 parameters. Don't worry about the last 2.
   • Close the window.

4. Now, expand the Single Particle Reconstruction item so you can see the detailed workflow under it.

Importing Particle Data

1. There are now 3 possibilities, for getting your data into EMAN2. You MUST reprocess the CTF in EMAN2, rather than use the already processed data from EMAN1 (unless you don't plan on doing any CTF correction), but on the bright side, this process is MUCH easier in EMAN2. So, the data we will import is the un phase-flipped data. It must also be completely unfiltered. Just the original, raw data. Pick only one of the following 3 sections in bold:

   • preferred option -> If you have the original micrographs/CCD frames, and EMAN1 style .box files :

     1. Select Filter Raw Data

     2. Press the Browse to Add button

     3. Browse to, and select your micrograph/CCD files, and click ok.

     4. Select which options you need to use, then press OK

        • It is important to get Inversion right. The final particles MUST appear bold white on a darker background, so whether you check this box depends on whether you are using stain or in cryo, and how the data was digitized. You can double-click on one of the images in the list to display it an see if you need inversion.

        • Filter X-ray Pixels is important for CCD data, but not if you are using a phase-plate.

        • edgenorm, generate thumbnail, and associate with project should all be checked

        • in-place processing should normally be unchecked.

     5. Under Particles, select Coordinate Import and Browse to Add again. Browse to and select your .box files, then press OK in both windows. Assuming your particle and box filenames match, everything should work well.

     6. You could actually go to the interactive particle picker and check/update the boxes at this point if you like, but we will just assume they are correct, and use Generate Output under Particles. Again, there are some options:

        • Box size is the most important one. In EMAN2, it is VERY important, if you want to obtain optimal results, to use a box size that is 1.5-2x the size of the maximum dimension of your particle. If you are dealing with a huge virus particle, you may need to skimp a bit on this, to avoid having a, say, 1500 box size, but some padding is essential for CTF correction to work well, and to avoid artifacts. See this list of good sizes to use. If you decide to change the box size later, it generally requires starting a new project from scratch, so consider this point very carefully.

        • You should select Write box image files, normalize.edgemean and bdb.

        • Only select invert here if you made a mistake at the earlier step, and your micrographs are still dark particles on a light background.

   • If you don't have those, but do have individual particle stack files without CTF correction for each frame :

     1. Double check to make sure your particle stacks have light particles on a darker background. If they have inverted contrast, you will need to invert them before proceeding. You can do this with e2proc2d.py <file> <file> --mult=-1 --inplace or any other method you like. Having this correct is CRITICAL to getting good results in EMAN2. Some of the algorithms require this.

     2. Select Particle Import under Particles, Browse to Add, and select all of your particle files. Press OK on both windows.

   • If you only have a start.hed file containing phase-flipped particles from EMAN1 :

     • Sorry, this section not complete yet

CTF Fitting/Correction (even for stain and phase plate)

1. Now that your particles have been imported into the EMAN2 project, you can begin with CTF correction. It is important to note at this point that CTF correction in EMAN2 entails a lot more than just CTF correction. It also makes an SSNR evaluation for each micrograph, which is used to perform weighting and other tasks. So, even if you do not feel you need to do CTF correction (neg stain or phase plates for example), we STRONGLY encourage you to go through this process anyway. You are always free to disable the CTF amplitude correction at a later stage, but you will still be able to take advantage of the other computed image properties.

2. Under CTF, select Automated Fitting. Select all of the image files (again, you can click in the upper left hand corner of the image list to select all quickly). Make sure the microscope parameters are correct. For Amplitude Contrast ~10-20 should be good for cryo, perhaps 50-70% for stain, and 100% for phase plate imaging. Set oversampling to 2 here for better defocus estimation (but make sure it's 1 when you generate output later). If you have a multi-core computer, enter the number of processors to use. Check Auto High Pass for normal cryo data, but not for negative stain or phase plate data. Note that autofitting typically requires a minimum of 20-30 particles per frame. If you have fewer particles than this, you may have to do a lot of manual defocus tuning later. When ready, press OK.

3. CTF fitting should not take long to complete. When it's done, select Interactive Tuning. At this point you do not need to check all of the images, but just a few you plan to use for structure factor determination. Click on the Particles on Disk column to sort in decreasing order. Then select the first 5-10 images in the list and click OK.

4. Three windows will appear: a control panel, a 2-D power spectrum view, and a 2-D plot. In this case all we need to do is make sure the defocus is roughly correct. This is the most common error in automatic CTF fitting. Either it will be completely correct, or the defocus will be way off. If you find an image with the wrong defocus, adjust it manually to roughly the correct position, and hit the refit button. If it is incorrect again after refit, then manually adjust to the correct value and Save Params. When you're done for all images you selected, close all 3 windows.

5. NOTE: If you are working with phase-plate data, hopefully your images will all have defocus of 0. The autofitting program is incapable of determining defocuses where fewer than 2 zeros appear in the power spectrum. So, for phase plate images, you will need to enter a defocus manually for all images.
6. Next, we need to generate a structure factor (you cannot presently use a structure factor from EMAN1). Again, sort the list of images by number of particles, and select the same images you manually checked in the previous step. Oversampling should be 2. Then press ok. The process should take only a few seconds. You can see the output on the console where you launched the workflow.
7. The generated structure factor goes into the project database, and a copy is written to a text file 'strucfac.txt'. You can plot this file vs any other structure factor to see how it compares if you wish, but it is not used for anything. The internal copy in the database is used for fitting and correction.
8. Repeat automatic CTF fitting for all images as you did it earlier. Now that you have a structure factor, the fitting should be more accurate.

9. Also run Interactive Tuning again. You are welcome to check all of the images, or just a few of them. If you check a few, and find that the fitting is poor, you should probably check the rest as well. If this happens, please email sludtke@bcm.edu, as such failures are valuable in improving the software. You may optionally go through the data and adjust the Quality slider to indicate relative quality of each image. This is an entirely user controlled parameter, and is not actually used directly by any of the programs. Note that you do NOT need to press the Save Parm button after adjusting the quality slider, as unlike the other parameters, it is autosaved.

10. Once you are happy with all of the CTF parameters, select Generate Output (under CTF). Make SURE that oversampling is set to 1 in this window. I suggest selecting all 3 checkboxes below. Click OK. This process can take a bit of time, and is limited by the speed of your hard drive.