ZFS Snapshots on TSCC

ZFS Overview

A ZFS snapshot is a read-only copy of a ZFS file system or volume. Snapshots can be created almost instantly and initially consume no additional disk space within the pool. All normal users on TSCC have access to their ZFS hidden file system at

$HOME/.zfs/

ZFS supports the ability to restore lost files with the standard UNIX copy command. See the example below.

Please note: Due to a ZFS bug, only administrators are currently able to access snapshots on TSCC. Please email TSCC User Support if you need to recover files from a ZFS-backed file system.

Read more about ZFS from Oracle. Learn about ZFS Snapshots.

The following conditions pertain to ZFS as implemented on TSCC:

The user account creation process automatically creates both a ZFS (home) file system and a Lustre PFS (Data Oasis) account for each user.
There are approximately 80TB of storage in the home file system, to be shared by all users.
Snapshots are taken daily and then replicated. This ensures local data areas have both readily accessible backups and duplicates in case of failures to the home system.
TSCC retains snapshot archives for approximately 30 days. This policy will continue as long as the pool retains sufficient space for daily needs of the community.
Users are requested to share the space responsibly, and to remove data in a timely fashion so the space becomes available for others.
Users are expected to make their own permanent backups of valuable data on the home file system. ZFS Snapshots are NOT permanent backups. Further, the length of time for which snapshots are retained is subject to change without prior notice.

Example of ZFS Data Snapshot Restore Session

[juser@login-4-0 ~]$ ls
1G  4G
[juser@login-4-0 ~]$ ls .zfs
snapshot/
[juser@login-4-0 ~]$ ls .zfs/snapshot
SNAPSHOT2009-06-22-1245668520/  SNAPSHOT2009-06-24-1245841320/
SNAPSHOT2009-06-22-1245668674/  SNAPSHOT2009-06-25-1245927720/
SNAPSHOT2009-06-23-1245754920/  SNAPSHOT2009-06-26-1246014120/
[juser@login-4-0 ~]$ rm 1G
[juser@login-4-0 ~]$ ls
4G
[juser@login-4-0 ~]$ ls .zfs/snapshot/SNAPSHOT2009-06-26-1246014120/
1G  4G
[juser@login-4-0 ~]$ cp .zfs/snapshot/SNAPSHOT2009-06-26-1246014120/1G .
[juser@login-4-0 ~]$ ls
1G  4G
[juser@login-4-0 ~]$

Cleanup and Reclamation of ZFS Storage

ZFS snapshots are periodically removed based on the above-mentioned policy. Large data applications sometimes have problems with availability of space for deleted files that are contained in snapshots, because the file space is not reclaimed until all snapshots that refer to a file have been deleted. ZFS must retain at least one snapshot to support incremental backups, or the system must copy the whole filesystem, which can take days.

In such cases, users can delete as many files as possible and then request for snapshots to be deleted. Administrators can then force the system to take a snapshot, allowing subsequent removal of the older snapshots that contain references to the deleted files.