|
Size: 10065
Comment:
|
← Revision 36 as of 2019-04-29 03:09:16 ⇥
Size: 2622
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| == Porting an EMAN1 refinement project to EMAN2 == | == Porting an EMAN1 refinement project to EMAN2.3 == |
| Line 3: | Line 3: |
| ''Note : This page is for those who hate to read. The [[EMAN2/Tutorials|tutorials]] provide a much better way of learning EMAN2. This will get you started if you just want to rapidly switch a project from EMAN1, though. Just remember if you find yourself asking 'but what does THAT mean' when you read this, you're reading the wrong page :^)'' |
''Note : This page contains a simplified version of the EMAN2.1 tutorial, and assumes you are familiar with EMAN1. The full [[EMAN2/Tutorials|tutorials]] provide a much better way of learning EMAN2. This page will get you started if you just want to rapidly switch a project from EMAN1. If you find yourself asking 'but what does THAT mean' when you read this, you're reading the wrong page!'' |
| Line 7: | Line 6: |
| === A Quickstart Guide === This quickstart makes use of the workflow, as the goal here is presumably to get the best results in EMAN2, and that is most easily accomplished in the workflow. It will take you all the way through a canonical reconstruction starting with data you've already processed in EMAN1 (or any other image processing package for that matter). This is written targeting primarily Linux/Mac users, but hopefully windows users will be able to follow along. |
=== A Quick (but worse) Alternative === Following the guide below will effectively start your project from an EMAN1 start.hed/img. There are a variety of advantages in doing things this way, as a number of things have improved in EMAN2 vs EMAN1. However, if your goal is just to get started quickly, even if the results are good, but perhaps not optimal, there is an alternative: |
| Line 11: | Line 9: |
| For purposes of this tutorial will call the directory where your eman1 data resides 'eman1'. You should also make another directory where your EMAN2 project will reside. We will call this 'eman2', but you can use any name you like. |
* create an empty folder adjacent to your EMAN1 refinement * run e2import.py --import_eman1 --curdefocushint ../folder/start.hed * note that the import script should work, but hasn't been updated in some time, so you will wind up with a few extra output files * run e2projectmanager.py and run the CTF -> CTF Autoprocess step (refer to the tutorial if you need details) |
| Line 15: | Line 14: |
| === Initial Setup === 1. It is very important that you ''cd eman2'' before starting the EMAN2 GUI. Whatever directory you are in when you run e2workflow.py will become your project directory. 1. Run ''e2workflow.py''. Two GUI windows should appear. One will show the status of running background jobs. The other will contain a collapsible tree of workflow items. 1. In the workflow window, click on the ''Single Particle Reconstruction'' item. * Note that you are not expanding the list here, but clicking on the actual words. * This should pop up a window containing project parameters. * Fill in the first 4 parameters. Don't worry about the last 2. * Close the window. 1. Now, expand the ''Single Particle Reconstruction'' item so you can see the detailed workflow under it. |
This will create a new EMAN2 project from start.hed/img. It will: * split the particles by defocus * undo the EMAN1 phase-flipping * redetermine the CTF using the EMAN1 parameters as a starting point (you can use --curdefocusfix as an alternative) * re-generate phase-flipped as well as filtered/shrunk particles * build sets containing all of the particles |
| Line 25: | Line 21: |
| === Importing Particle Data === 1. There are now 3 possibilities, for getting your data into EMAN2. You MUST reprocess the CTF in EMAN2, rather than use the already processed data from EMAN1 (unless you don't plan on doing any CTF correction), but on the bright side, this process is MUCH easier in EMAN2. So, the data we will import is the un phase-flipped data. It must also be completely unfiltered. Just the original, raw data. Pick only one of the following 3 sections in bold: * '''preferred option -> If you have the original micrographs/CCD frames, and EMAN1 style ''.box'' files ''': 1. Select ''Filter Raw Data'' 1. Press the ''Browse to Add'' button 1. Browse to, and select your micrograph/CCD files, and click ok. 1. Select which options you need to use, then press ''OK'' * It is important to get ''Inversion'' right. The final particles MUST appear '''bold''' white on a darker background, so whether you check this box depends on whether you are using stain or in cryo, and how the data was digitized. You can double-click on one of the images in the list to display it an see if you need inversion. * ''Filter X-ray Pixels'' is important for CCD data, but not if you are using a phase-plate. * ''edgenorm, generate thumbnail'', and ''associate with project'' should all be checked * ''in-place processing'' should normally be unchecked. 1. Under ''Particles'', select ''Coordinate Import'' and ''Browse to Add'' again. Browse to and select your ''.box'' files, then press ''OK'' in both windows. Assuming your particle and box filenames match, everything should work well. 1. You could actually go to the interactive particle picker and check/update the boxes at this point if you like, but we will just assume they are correct, and use ''Generate Output'' under Particles. Again, there are some options: * Box size is the most important one. In EMAN2, it is VERY important, if you want to obtain optimal results, to use a box size that is 1.5-2x the size of the maximum dimension of your particle. If you are dealing with a huge virus particle, you may need to skimp a bit on this, to avoid having a, say, 1500 box size, but some padding is essential for CTF correction to work well, and to avoid artifacts. See [[EMAN2/BoxSize|this list]] of good sizes to use. If you decide to change the box size later, it generally requires starting a new project from scratch, so consider this point very carefully. * You should select ''Write box image files'', ''normalize.edgemean'' and ''bdb''. * Only select ''invert'' here if you made a mistake at the earlier step, and your micrographs are still dark particles on a light background. * '''If you don't have those, but do have individual particle stack files without CTF correction for each frame :''' 1. Double check to make sure your particle stacks have light particles on a darker background. If they have inverted contrast, you will need to invert them before proceeding. You can do this with ''e2proc2d.py <file> <file> --mult=-1 --inplace'' or any other method you like. Having this correct is CRITICAL to getting good results in EMAN2. Some of the algorithms require this. 1. Select ''Particle Import'' under ''Particles'', ''Browse to Add'', and select all of your particle files. Press ''OK'' on both windows. * '''If you only have a ''start.hed'' file containing phase-flipped particles from EMAN1 :''' * '''''Sorry, this section not complete yet''''' === CTF Fitting/Correction (even for stain and phase plate) === 1. Now that your particles have been imported into the EMAN2 project, you can begin with CTF correction. It is important to note at this point that CTF correction in EMAN2 entails a lot more than just CTF correction. It also makes an SSNR evaluation for each micrograph, which is used to perform weighting and other tasks. So, even if you do not feel you need to do CTF correction (neg stain or phase plates for example), we STRONGLY encourage you to go through this process anyway. You are always free to disable the CTF amplitude correction at a later stage, but you will still be able to take advantage of the other computed image properties. 1. Under CTF, select ''Automated Fitting''. Select all of the image files (again, you can click in the upper left hand corner of the image list to select all quickly). Make sure the microscope parameters are correct. For ''Amplitude Contrast'' ~10-20 should be good for cryo, perhaps 50-70% for stain, and 100% for phase plate imaging. Set oversampling to 2 here for better defocus estimation (but make sure it's 1 when you generate output later). If you have a multi-core computer, enter the number of processors to use. Check ''Auto High Pass'' for normal cryo data, but not for negative stain or phase plate data. Note that autofitting typically requires a minimum of 20-30 particles per frame. If you have fewer particles than this, you may have to do a lot of manual defocus tuning later. When ready, press ''OK''. 1. CTF fitting should not take long to complete. When it's done, select ''Interactive Tuning''. At this point you do not need to check all of the images, but just a few you plan to use for structure factor determination. Click on the ''Particles on Disk'' column to sort in decreasing order. Then select the first 5-10 images in the list and click ''OK''. 1. Three windows will appear: a control panel, a 2-D power spectrum view, and a 2-D plot. In this case all we need to do is make sure the defocus is roughly correct. This is the most common error in automatic CTF fitting. Either it will be completely correct, or the defocus will be way off. If you find an image with the wrong defocus, adjust it manually to roughly the correct position, and hit the ''refit'' button. If it is incorrect again after ''refit'', then manually adjust to the correct value and ''Save Params''. When you're done for all images you selected, close all 3 windows. 1. NOTE: If you are working with phase-plate data, hopefully your images will all have defocus of 0. The autofitting program is incapable of determining defocuses where fewer than 2 zeros appear in the power spectrum. So, for phase plate images, you will need to enter a defocus manually for all images. 1. Next, we need to generate a structure factor (you cannot presently use a structure factor from EMAN1). Again, sort the list of images by number of particles, and select the same images you manually checked in the previous step. Oversampling should be 2. Then press ok. The process should take only a few seconds. You can see the output on the console where you launched the workflow. 1. The generated structure factor goes into the project database, and a copy is written to a text file 'strucfac.txt'. You can plot this file vs any other structure factor to see how it compares if you wish, but it is not used for anything. The internal copy in the database is used for fitting and correction. 1. Repeat automatic CTF fitting for all images as you did it earlier. Now that you have a structure factor, the fitting should be more accurate. 1. Also run ''Interactive Tuning'' again. You are welcome to check all of the images, or just a few of them. If you check a few, and find that the fitting is poor, you should probably check the rest as well. If this happens, please email '''sludtke@bcm.edu''', as such failures are valuable in improving the software. You may optionally go through the data and adjust the ''Quality'' slider to indicate relative quality of each image. This is an entirely user controlled parameter, and is not actually used directly by any of the programs. Note that you do NOT need to press the ''Save Parm'' button after adjusting the quality slider, as unlike the other parameters, it is autosaved. 1. Once you are happy with all of the CTF parameters, select ''Generate Output'' (under CTF). Make SURE that ''oversampling'' is set to 1 in this window. I suggest selecting all 3 checkboxes below. Click ''OK''. This process can take a bit of time, and is limited by the speed of your hard drive. |
That's it, you're ready to start a refinement, though results may not be optimal. ---- While the above process should get you started quickly, to get optimal results, you should really follow the full single particle processing [[EMAN2/Tutorials|tutorial]], starting from micrographs, with the following caveats: * When importing micrographs, make sure you get the inversion correct. Particles in EMAN2 should be light on a darker background just like in EMAN1, and it is best if you start with the micrographs in this state. * You can use the import .box file tool just like in the tutorial * EMAN2 is much more sensitive to having a sufficiently large [[EMAN2/BoxSize|box size]. Before doing the "Particles -> Generate Output" step, make sure the box size you plan to use is adequete. * While there is no harm in doing unsupervised class-averaging, you probably already have a good 3-D starting model from your EMAN1 work, so you can likely skip the 2-D class averaging and initial model generation steps if you like |
Porting an EMAN1 refinement project to EMAN2.3
Note : This page contains a simplified version of the EMAN2.1 tutorial, and assumes you are familiar with EMAN1. The full tutorials provide a much better way of learning EMAN2. This page will get you started if you just want to rapidly switch a project from EMAN1. If you find yourself asking 'but what does THAT mean' when you read this, you're reading the wrong page!
A Quick (but worse) Alternative
Following the guide below will effectively start your project from an EMAN1 start.hed/img. There are a variety of advantages in doing things this way, as a number of things have improved in EMAN2 vs EMAN1. However, if your goal is just to get started quickly, even if the results are good, but perhaps not optimal, there is an alternative:
- create an empty folder adjacent to your EMAN1 refinement
- run e2import.py --import_eman1 --curdefocushint ../folder/start.hed
- note that the import script should work, but hasn't been updated in some time, so you will wind up with a few extra output files
run e2projectmanager.py and run the CTF -> CTF Autoprocess step (refer to the tutorial if you need details)
This will create a new EMAN2 project from start.hed/img. It will:
- split the particles by defocus
- undo the EMAN1 phase-flipping
- redetermine the CTF using the EMAN1 parameters as a starting point (you can use --curdefocusfix as an alternative)
- re-generate phase-flipped as well as filtered/shrunk particles
- build sets containing all of the particles
That's it, you're ready to start a refinement, though results may not be optimal.
While the above process should get you started quickly, to get optimal results, you should really follow the full single particle processing tutorial, starting from micrographs, with the following caveats:
- When importing micrographs, make sure you get the inversion correct. Particles in EMAN2 should be light on a darker background just like in EMAN1, and it is best if you start with the micrographs in this state.
- You can use the import .box file tool just like in the tutorial
EMAN2 is much more sensitive to having a sufficiently large [[EMAN2/BoxSize|box size]. Before doing the "Particles -> Generate Output" step, make sure the box size you plan to use is adequete.
- While there is no harm in doing unsupervised class-averaging, you probably already have a good 3-D starting model from your EMAN1 work, so you can likely skip the 2-D class averaging and initial model generation steps if you like