== Folder Arrangement in EMAN2.1 projects == When using e2projectmanager.py and following canonical EMAN2.1 procedures, your data will be contained within a "project" with a very specific organization. For things to work properly you must not break this organization. If you are just running EMAN2 command-line programs yourself, and are not using the projectmanager or other GUI tools, you can do what you like, of course, but regardless we strongly suggest following the canonical structure. '''''Note:''''' '''It is critical when running command-line programs within a project that you run them from the project folder, not from subfolders. For example, if you are in the ''particles'' folder, then try to build a set by referencing ''../sets'' you can create all sorts of havoc.''' A project directory will normally contain these folders, and some other (unlisted) files: {{{ info/ particles/ micrographs/ (optional) sets/ multi_xx r2d_xx refine_xx }}} The particles/ folder will contain files like: {{{ 01252014_AE_100__ctf_flip.hdf 01252014_AE_100__ctf_flip_hp.hdf 01252014_AE_100__ctf_wiener.hdf 01252014_AE_100_ptcls.hdf 01252014_AE_101__ctf_flip.hdf 01252014_AE_101__ctf_flip_hp.hdf 01252014_AE_101__ctf_wiener.hdf 01252014_AE_101_ptcls.hdf }}} where the __ (double underscore) denotes modifications of the same set of particles. The ''info/'' folder will then contain files like: {{{ 01252014_AE_100_info.json 01252014_AE_101_info.json 01252014_AE_102_info.json }}} Where each file contains the information for a particular micrograph as documented [[http://blake.bcm.edu/emanwiki/Eman2InfoMetadata||on the JSON page]]. These JSON files are generally human-readable and editable, though sometimes they will contain text-encoded binary data which can make parts of them difficult to read. Sets/ will contain files like: {{{ my_combine__ctf_flip_hp.lst my_combine__ctf_flip.lst my_combine__ctf_wiener.lst my_combine_ptcls.lst }}} and the contents of one of these ''.lst'' files will look like: {{{ #LSX # This file is in fast LST format. All lines after the next line have exactly the number of characters shown on the next line. This MUST be preserved if editing. # 47 0 particles/01252014_AE_100_ptcls.hdf 1 particles/01252014_AE_100_ptcls.hdf 2 particles/01252014_AE_100_ptcls.hdf 3 particles/01252014_AE_100_ptcls.hdf 4 particles/01252014_AE_100_ptcls.hdf 5 particles/01252014_AE_100_ptcls.hdf }}} Note that the references to individual particles are from the project folder. LST files are just text files containing references to particles in other image files, but can be treated as if the were actually the image files themselves. That is, you can run ''e2proc2d.py'' on an LST file just as if it actually contained the images it references. Hand-editing LSX files is not recommended, since every line must have exactly the same number of characters in it, or the file will become invalid. === Why this structure ? === This structure is not arbitrary, and its logic has been carefully designed. If, for example, LST files contained an absolute path to a referenced image file, like ''/home/stevel/data/particles/abc.hdf'', then if I later decided to move my project to a different hard drive, the LST file would no longer work properly. Similarly, if the LST files in ''sets'' contained references to ''../particles/abc.hdf'', it would be confusing because that means you would have to run programs from within the ''sets'' directory for the references to be valid. To deal with these and other issues, the overall structure is such that all references are made relative to the project folder, and it is expected that command-line programs will all be executed from the project folder. This makes the project the main organizational unit for data. A project folder can be moved around from disk to disk or machine to machine without anything breaking. Additional sub-folders can be made within a project, as long as the rules are followed, and programs are run from the project level. That is, with this scheme, you never have to ask yourself, "now what folder should I be in when I run my refinement?" The answer is always "the project folder".