Every Thursday, from 1 to 2pm, the Hummingbird team is drop-in open-office hours to help you with your technical questions and issues. Need help with your SLURM Script? Want to know how to transfer files to your collaborators? Want to start using Git to manage your research code? Stop by – we’re happy to help!
https://ucsc.zoom.us/j/93463299124?pwd=RFZvZzlYSjIzNnoxblFsRUV6aTZGZz09
Add the weekly series to your calendar
SAFEGUARD AGAINST MISSING DATA
COPY versus MOVE
It has come to our attention that some users who are moving large chunks of data between locations on Hummingbird (e.g. from one directory to another) have unexpectedly seen the data disappear from their directories after the move has completed. This behavior seems to be associated only with moving TBs of data, at least more than just hundreds of gigabytes, but not smaller chunks of data.
The recommended solution is to COPY data (using the command rsync or cp) and NOT move the data (using the command mv). Once you have successfully copied the data, please check to see that the metadata is the same (e.g. the file size in bits), before you remove the old data from their original location.
If you observe that you have used MOVE and data are missing, we have no way to recover the lost data. This is a reminder that you should always have a copy of the data off Hummingbird for anything critical. We recognize that it’s not practical to make copies of everything on systems other than Hummingbird, but for final results and critical, non-intermediate data, this is good practice.
We will continue to investigate this problematic behavior, but your assistance by using COPY for moving large files or directories will help ensure that your data remain intact and available to you.
UC Santa Cruz Research Computing
