e2refine2d.py runs in much the same way as EMAN1's refine2d.py, though it has been improved in a number of subtle ways

This program will take a set of boxed out particle images and perform iterative reference-free classification to produce a set of representative class-averages. The point of this process is to reduce noise levels, so the overall shape of the particle views present in the data can be better observed. Generally cryo-EM single particles are noisy enough that it is difficult to distinguish subtle, or even not-so-subtle differences between particle images. By aligning and averaging similar particles together, less noisy versions of representative views are created. The class-averages produced by this program are typically used for:

This last point can be used to produce 'population-dynamics' movies of a particle in very close to the same orientation.

This program is quite fast for as many as a few thousand particles and ~100 classes. For most purposes if your data set is large (>10,000) particles you might consider using only a subset of the data for speed, though this clearly isn't appropriate for the 3rd use above. For large numbers of classes, specify the --fastseed option, or you will wait a very long time.




Path for the refinement, default=auto



The total number of refinement iterations to perform



This will perform a 2-D automask on class-averages to help with centering. May be useful for negative stain data particularly.



The name of the file containing the particle data



Number of classes to generate



Maximum particle translation in x and y



Number of alignment references to when determining particle orientations



The named file should contain a set of integers, each representing an image from the input file to exclude.



This will cause a check of the files in the current directory, and the refinement will resume after the last completed iteration. It's ok to alter other parameters.



File containing starting class-averages. If not specified, will generate starting averages automatically



Number of MSA basis vectors to use when classifying particles



Minimum number of particles that change group before deicding to terminate. Default = -1 (auto)



Will seed the k-means loop quickly, but may produce less consistent results.



The name of an 'aligner' to use prior to comparing the images (default=rotate_translate_flip)



Name of the aligner along with its construction arguments (default=frc)



The name and parameters of the second stage aligner which refines the results of the first alignment



The name and parameters of the comparitor used by the second stage aligner. (default=dot).



The name of a 'cmp' to be used in comparing the aligned images (default=frc:nweight=1)



Optionally shrink the input particles by an integer amount prior to computing similarity scores. For speed purposes.



The fraction of particles to keep in each class, based on the similarity score generated by the --cmp argument (default=0.85).



Change the keep ('--keep') criterion from fraction-based to sigma-based.



Number of iterations to use when making class-averages (default=5)



If doing more than one iteration, this is the name and parameters of the 'aligner' used to align particles to the previous class average.



This is the name and parameters of the comparitor used by the fist stage aligner Default is dot.



The second stage aligner which refines the results of the first alignment in class averaging. Default is None.



The comparitor used by the second stage aligner in class averageing. Default is dot:normalize=1.



The averager used to generate the class averages. Default is 'mean'.



The name and parameters of the comparitor used to generate similarity scores, when class averaging. Default is frc'



Normalization applied during class averaging



Use the setsfref option in class averaging to produce better filtered averages.



Normalizes each projected vector into the MSA subspace. Note that this is different from normalizing the input images since the subspace is not expected to fully span the image




Run in parallel, specify type:<option>=<value>:<option>:<value>



data base list storage, used by the workflow. You can ignore this argument.




verbose level [0-9], higner number means higher level of verboseness

Typical usage:

e2refine2d.py --iter=6 --naliref=7 --nbasisfp=5 --path=r2d_01 --input=inputparticles.hdf --ncls=48 --simcmp=ccc --simalign=rotate_translate_flip --simaligncmp=ccc --simralign=refine --simraligncmp=ccc --classcmp=ccc --classalign=rotate_translate_flip --classaligncmp=ccc --classralign=refine --classraligncmp=ccc --classiter=2 --classkeep=1.5 --classnormproc=normalize.edgemean --classaverager=mean --normproj --classkeepsig --parallel=thread:4

Primary options to consider changing:

Of course, all of the other parameters are meaningful as well, but the ones listed above are the ones you would change first if you don't get the results you expect.

This program uses an iterative MSA-based reference-free classification algorithm. The names in parentheses below are the filenames produced by each step. The files will be found in bdb:r2d_XX (XX is incremented each time e2refine2d.py is run). A brief outline of the process follows :

  1. Initialize the iterative process by making some initial guesses at class-averages. These are invariant-based, meaning even with MSA, this initial classification is not exceptionally good.
    1. Generate rotational/translational invariants for each particle (input_fp)
    2. Perform MSA on the invariants to define an orthogonal subspace representing the most important differences among the classes (input_fp_basis)
    3. Reproject the particles into the MSA subspace using --nbasis vectors (input_fp_basis_proj)

    4. Classify the particles into --ncls classes using K-means (classmx_00)

    5. Iterative class-averaging of the particles in each class to produce a set of initial averages (classes_init)
  2. Align the current class-averages to each other, and sort them by similarity, keeping them centered (allrefs_YY) (Note that YY starts with 01 and is incremented after each iteration)
  3. Perform MSA on the (aligned) class-averages. Again, this represents largest differences, but now performed on images, not invariants. (basis_YY)
  4. Select a subset of --naliref averages to use as alignment references for this iteration (aliref_YY)

  5. Align each particle to each of the reference averages from the last step. Keep the orientation corresponding to the best-matching reference. (simmx_YY)
  6. Project aligned particles using reference MSA vectors from basis_YY (input_YY_proj)
  7. K-means classification of input_YY_proj (classmx_YY)
  8. New iterative class-averages (classes_YY)
  9. Loop back to step 2 until --iter loops are complete

The primary files you would normally look at after a run is complete are:

EMAN2/Programs/e2refine2d (last edited 2012-04-30 19:57:01 by SteveLudtke)