|
Size: 2808
Comment:
|
← Revision 45 as of 2023-07-19 17:01:38 ⇥
Size: 5450
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = CIBR and other shared clusters = | = CIBR Cluster Co-op = |
| Line 3: | Line 3: |
| [[https://www.bcm.edu/research/centers/computational-and-integrative-biomedical-research/|CIBR]] and a number of specific faculty have sponsored the purchase of several clusters managed for shared use by faculty. CIBR faculty may request moderate allocations for free, as a benefit of CIBR membership (which is free). Those requiring larger resources or dedicated time can participate in the purchase of future cluster equipment, and will get access to corresponding resources for the life of the cluster. | '''WARNING: ''' The existing CIBR clusters are now ~10 years old. While still functioning properly, apart from a few retired nodes, they are getting old enough that the possibility of a fatal failure is eventually inevitable. Once the clusters die to a sufficient degree that they cannot be cheaply repaired, they will be retired. This may happen without warning, so it is critical that you maintain a local copy of any important data. Note that none of the RAID arrays, including store1, are backed up any more, due to the total failure of store2, so all storage is one major failure away from retirement. |
| Line 5: | Line 5: |
| The 4 clusters currently in operation are: | ''Please note that the CIBR cluster Co-op strives to maximize CPU/cost. On most clusters, staff support is one of the largest expenses. So, while we routinely support issues with hardware, queuing system, operating system problems, etc. we have absolutely no applications support staff. At present, we actually have no BCM support for cluster support staff at all, and Dr. Ludtke's group is handling issues as a community service.'' [[https://www.bcm.edu/research/centers/computational-integrative-biomedical-research|CIBR]] and a number of specific faculty have sponsored the purchase of several clusters managed for shared use by faculty. CIBR faculty may request moderate allocations for free, as a benefit of CIBR membership (which is free). Faculty who contributed funds directly to the purchase have larger guaranteed allocations. The clusters currently in operation are: |
| Line 7: | Line 13: |
| ||[[CIBRClusters/Sphere|sphere.grid.bcm.edu]]||960||60 TB + 80 TB||2015||25,000 CPU-hr/qtr||Chiu, Barth, Guan, Waterland || Now available for open testing!|| ||[[CIBRClusters/Prism|prism.grid.bcm.edu]]||704||63+88 TB||2013||25,000 CPU-hr/qtr||Chiu, Barth, Wensel, Ludtke, Guan|| || ||[[CIBRClusters/Torus|torus.grid.bcm.edu]]||576+64||40+48+25 TB||2011-2012||10,000 CPU-hr/qtr||Chiu, Ludtke, Barth||Has 4 special 'bioinformatics' nodes with 16 cores and dedicated high performance storage|| ||asterix.grid.bcm.edu||180+42 GPUs||10 TB||2009-2011||None||Ludtke, Chiu||GPUs, special projects, NCMI personnel can contact Dr. Ludtke|| ||store1|| - || 350 TB Raid6x2 || 2013 || variable || - || Reliable storage for inactive data || |
||[[CIBRClusters/Sphere|sphere.grid.bcm.edu]]||960||60 TB + 80 TB||2015||25,000 CPU-hr/qtr||Ludtke, Guan, Waterland || || ||[[CIBRClusters/Prism|prism.grid.bcm.edu]]||704||63+88 TB||2013||25,000 CPU-hr/qtr|| Wensel, Ludtke, Guan|| || ||[[CIBRClusters/Store|store]]|| - || 350 TB Raid6x2 || 2013 || upon request || - || For inactive data which will still be needed on the clusters. This storage array is available upon request to users of any of the clusters, but is directly accessible only from the clusters to discourage people for using it for general lab backup purposes. It is a RAID6 volume, which gives it good reliability, but every 4-5 years we experience some sort of failure. At one time this storage was backed up in addition to being a RAID6 array, but that is no longer true. If the RAID suffers a critical failure, all data could be lost. || |
| Line 13: | Line 17: |
| === Requesting Access === Free cluster accounts must be requested directly by a PI (tenure track faculty) who must be an established member of CIBR, and such accounts will charge against that PI's allocation. Contact Richard.Sucgang@bcm.edu about CIBR membership. We may ask to discuss your intended use for the cluster before granting accounts, simply to insure that it will not interfere with paid cluster users or violate policy. Please note that cluster security does not meet HIPAA requirements, so no identified patient data may be stored or processed on the clusters. To request an account, email sludtke@bcm.edu with: * Full name * Requested username * Position (student, postdoc, etc) * Cell Phone Number (for emergencies and initial password) * An email address which must be checked at least daily. === Cluster Usage === |
|
| Line 15: | Line 30: |
| All of the clusters run some version of CentOS Linux. We do not provide any commercial software, but most standard open-source tools are installed, and users are welcome to install commercial or free software within their own accounts. BCM's site license for Matlab is usable on the cluster, but must be installed/licensed in the user's account. Contact Larry Mayran about this for details. We may also be able to install other software system-wide, if you have such a need, please ask. | Individual users are expected to be familiar with the use of Linux clusters in general, and if not, to obtain assistance from their acquaintances, rather than learning by trial and error. Occasionally CIBR offers a workshop to help familiarize people with cluster computing, which will be advertised via normal CIBR mechanisms. |
| Line 17: | Line 32: |
| Details on some of the open-source software we have made available for users is maintained on the [[CIBRClusters/Software|Software page]]. | All of the clusters run some version of CentOS Linux. '''We do not provide any commercial software''', but most standard open-source tools are installed, and users are welcome to install commercial or free software within their own accounts. BCM's site license for Matlab is usable on the cluster, but must be installed/licensed in the user's account, and there are some setup issues. Contact Larry Mayran about this for details. We may also be able to install other open-source software system-wide, if you have such a need, please ask. |
| Line 19: | Line 34: |
| '''In manuscripts, grant proposals, presentations, and other publications, please acknowledge the assistance from the CIBR Center by including a statement similar to the following:''' | Details on some of the open-source software we have made available for users is on the [[CIBRClusters/Software|Software page]]. There is a CIBR Cluster Google Group/Mailing List (https://groups.google.com/forum/#!forum/cibrcluster) which is used for announcements of outages, problems, policy changes, etc. We strongly recommend all CIBR Cluster users join. === In manuscripts, grant proposals, presentations, and other publications === Please acknowledge the assistance from the CIBR Center by including a statement similar to the following: |
| Line 23: | Line 43: |
| CIBR recently hosted a Mini-Workshop on cluster computing. The slides can be downloaded here: [[attachment:workshop2015.pdf]] | CIBR hosted a Mini-Workshop on cluster computing in 2018: The slides can be downloaded here: [[attachment:workshop2018.pdf]] Archived video of the presentation is here: [[http://blake.grid.bcm.edu/dl/CIBR/cluster_computing.mp4|Cluster Computing]] |
CIBR Cluster Co-op
WARNING: The existing CIBR clusters are now ~10 years old. While still functioning properly, apart from a few retired nodes, they are getting old enough that the possibility of a fatal failure is eventually inevitable. Once the clusters die to a sufficient degree that they cannot be cheaply repaired, they will be retired. This may happen without warning, so it is critical that you maintain a local copy of any important data. Note that none of the RAID arrays, including store1, are backed up any more, due to the total failure of store2, so all storage is one major failure away from retirement.
Please note that the CIBR cluster Co-op strives to maximize CPU/cost. On most clusters, staff support is one of the largest expenses. So, while we routinely support issues with hardware, queuing system, operating system problems, etc. we have absolutely no applications support staff. At present, we actually have no BCM support for cluster support staff at all, and Dr. Ludtke's group is handling issues as a community service.
CIBR and a number of specific faculty have sponsored the purchase of several clusters managed for shared use by faculty. CIBR faculty may request moderate allocations for free, as a benefit of CIBR membership (which is free). Faculty who contributed funds directly to the purchase have larger guaranteed allocations.
The clusters currently in operation are:
Name |
Cores |
Storage |
Purchased |
Free CIBR Access |
Paid Access |
Notes |
960 |
60 TB + 80 TB |
2015 |
25,000 CPU-hr/qtr |
Ludtke, Guan, Waterland |
|
|
704 |
63+88 TB |
2013 |
25,000 CPU-hr/qtr |
Wensel, Ludtke, Guan |
|
|
- |
350 TB Raid6x2 |
2013 |
upon request |
- |
For inactive data which will still be needed on the clusters. This storage array is available upon request to users of any of the clusters, but is directly accessible only from the clusters to discourage people for using it for general lab backup purposes. It is a RAID6 volume, which gives it good reliability, but every 4-5 years we experience some sort of failure. At one time this storage was backed up in addition to being a RAID6 array, but that is no longer true. If the RAID suffers a critical failure, all data could be lost. |
Requesting Access
Free cluster accounts must be requested directly by a PI (tenure track faculty) who must be an established member of CIBR, and such accounts will charge against that PI's allocation. Contact Richard.Sucgang@bcm.edu about CIBR membership. We may ask to discuss your intended use for the cluster before granting accounts, simply to insure that it will not interfere with paid cluster users or violate policy. Please note that cluster security does not meet HIPAA requirements, so no identified patient data may be stored or processed on the clusters.
To request an account, email sludtke@bcm.edu with:
- Full name
- Requested username
- Position (student, postdoc, etc)
- Cell Phone Number (for emergencies and initial password)
- An email address which must be checked at least daily.
Cluster Usage
Each cluster has independent user accounts, as different faculty contributed different amounts to each. Please click on one of the links above for detailed policies and procedures for each cluster
Individual users are expected to be familiar with the use of Linux clusters in general, and if not, to obtain assistance from their acquaintances, rather than learning by trial and error. Occasionally CIBR offers a workshop to help familiarize people with cluster computing, which will be advertised via normal CIBR mechanisms.
All of the clusters run some version of CentOS Linux. We do not provide any commercial software, but most standard open-source tools are installed, and users are welcome to install commercial or free software within their own accounts. BCM's site license for Matlab is usable on the cluster, but must be installed/licensed in the user's account, and there are some setup issues. Contact Larry Mayran about this for details. We may also be able to install other open-source software system-wide, if you have such a need, please ask.
Details on some of the open-source software we have made available for users is on the Software page.
There is a CIBR Cluster Google Group/Mailing List (https://groups.google.com/forum/#!forum/cibrcluster) which is used for announcements of outages, problems, policy changes, etc. We strongly recommend all CIBR Cluster users join.
In manuscripts, grant proposals, presentations, and other publications
Please acknowledge the assistance from the CIBR Center by including a statement similar to the following: "We gratefully acknowledge the assistance and computing resources provided by the CIBR Center for Computational and Integrative Biomedical Research of Baylor College of Medicine in the completion of this work."
CIBR hosted a Mini-Workshop on cluster computing in 2018: The slides can be downloaded here: workshop2018.pdf Archived video of the presentation is here: Cluster Computing