Your Home Directory
Your home directory is for your day to day, non-computational work, papers, software development, administrative documents, etc., etc. Basic statistical work can run in your home directory fine, but under no circumstances should you run intensive batch jobs or condor out of your home directory. That puts a heavy load on the file server and will make life agonizing for most people in the department. We have separate research spaces for this sort of work (see below).
Backups
We do not use tape backup for any of our file storage. Instead, we use a system called ZFS as well as a replication infrastructure which keeps a copy of everything offsite. The most useful part of ZFS for the average person is the periodic snapshots — a sort of live backup — that you can access easily on your own. From OSX or Linux, you just need to go to the directory ~/.zfs/snapshots and pick a directory with the date you want. A copy of your home directory at that time will appear in the snapshot directory. Just copy the files you want.
Windows users: to mount your home directory and project directories see How to map a samba drive.
Your Research Data
Your home directory is not an appropriate place for collecting the output from intensive computational work — it slows down the server for everyone else. We have separate file servers for both computational work as well as for long-term storage of inactive data (archives, essentially).
We also recommend that labs with multiple people working on a single project get project directories. That way, work for particular project is in a central location. It avoids problems when people graduate and research has to be moved around to be saved.
Please fill out this form to set up a new computational space for your research – https://forms.biostat.wisc.edu/jira-mailer/project-directory
Seeing your Project and Computational Directories
When you get a project directory, it will have a name like /z/Proj/your-project. Many people have the common Unix workflow where they will cd to successive steps in the directory path to get somewhere, using ls to make sure they’re in the right place:
$ cd /z
$ ls
Comp/ DW/ Proj/ Users
$ cd Proj
$ ls
$