Supporting Infrastructure¶
Storage Retention Policies¶
Summit¶
| Content | Retention (days) | Provider | Server |
|---|---|---|---|
| AuxTel Raw Images (Butler) | 90 | Ceph NFS | yagan |
| AuxTel Quicklook Images (Butler) | ?? (90) | Ceph NFS | yagan |
| Large File Annex | 30 | Ceph S3 | chonchon |
| RubinTV Products | ?? | Ceph S3 | chonchon |
| ComCam Raw Images (Butler) | 90 | NFS | comcam-archiver |
| ComCam Quicklook Images (Butler) | ?? (90) | NFS | comcam-archiver |
| LSSTCam Raw Images (Butler) | ?? | Ceph NFS | chonchon |
| LSSTCam Quicklook Images (Butler) | ?? | Ceph NFS | chonchon |
| LSSTCam Raw Images (Butler) | 30 | Ceph S3 | new cluster |
| LSSTCam Quicklook Images (Butler) | ?? (30) | Ceph S3 | new cluster |
| DREAM Products | ?? | Ceph S3 | new cluster |
| EFD | 30 | Ceph PVC | yagan |
- Notes:
- Quicklook images are generated by Rapid Analysis.
- Butler retentions are shared, but independent application is in progress.
- The Ceph S3 on the new cluster entries are for future reference. System is currently not ready for use.
- Open issues:
- All retention policies should be reviewed for relevance.
- Anything with ?? needs a decision.
- Is DREAM going to store any products when the new cluster is available?
- Since calibration products are absolved from deletion, they will accumulate over time. How do we decide when and/or if to purge?
- User data products in the butler are not covered by a retention policy. Should they?
- Ceph NFS for LSSTCam and ComCam will have to be shared so will greatly influence the retention duration. The other option is to keep ComCam on the current storage and allocate the emergency storage all to LSSTCam.