HOW TO RECOVER DELETED FILES IN LINUX

Have you ever gotten that horrible feeling? The one you get when you realize that you accidentally deleted files and it’s not even in the trash? Often it is immediately preceded by denial: I know I have another copy of it somewhere.

But rather than going through all the stages of grief, don’t worry. And remember you’re not alone; sooner or later everyone does this.

“Don’t worry?” you counter, “I just erased the only copy of my resume!”

No really, don’t worry. All that’s happened is that it’s been bumped off a list. So long as you don’t write onto the drive, it absolutely still exists. In fact, depending on the size of the file and the free space on your drive deleted files can persist indefinitely—even if you do write on the drive.

“Yes, fine” you say, “I’ll rest easy knowing my resume ‘exists’ in some abstract sense. But so far as I’m concerned if I can’t open, edit or print from it, it doesn’t exist in any practical sense. What would really help would be a way to ‘un-delete’ files. And one that doesn’t require an IT forensics lab.”

Really, don’t worry—you don’t need a lab to recover the deleted files. Furthermore, if you can get past using a primitive GUI, it’s actually easy to do! I’ll show you how to use TestDisk to recover deleted files.

HOW TO RECOVER DELETED FILES IN LINUX USING TESTDISK

How to recover deleted files in Linux

Let me present a simplified example: I took a clean thumb drive added some files, then deleted one. Now, my system has a feature which will directly delete files from removable media, by-passing the “trash” altogether; that is if I choose to “right” click on a file and then choose “delete”. It still presents a warning, but one click on the “yes” button and the file is gone forever. Or appears to be.

But this time I didn’t get that horrible feeling. And no, not because this is a cooked up scenario. I knew that all I had to do was open the terminal type “testdisk” and hit “enter”. When I did this for the first time I had one of my “Linux moments”. Because if you don’t have it—and I didn’t—it tells you how to get it! Just type “sudo apt install testdisk” and enter and you’ll have it in about 10 seconds.

STEP 1

You need to install TestDisk tool first. Most Linux distributions already have this tool in their official repository. In Ubuntu and other Ubuntu based Linux distributions such as Linux Mint, elementary OS etc, you can use the command below to install TestDisk:

sudo apt install testdisk

Arch Linux users can install it from AUR. You can download it for other Linux distributions from the link below:

Download TestDisk

Though I am using Ubuntu in this tutorial, this doesn’t mean it is only to recover deleted files in Ubuntu Linux. The instructions presented here works for other distributions as well.

STEP 2

Run TestDisk in the terminal using the command below:

testdisk

STEP 3

When you open it, you’ll see something that looks like this. Be patient! The interface is actually straightforward but you do have to carefully read the text. Use the arrow keys to navigate and “enter” to select.

How to recover deleted files in Linux using TestDisk
Select ‘Create a new log file’

Screens that have extra commands will tell you so. Also note that TestDisk 7.0 tends to highlight the next reasonable step. It’s almost always right but do read the screen, since it can’t read your mind. In any case, when it wants you to let it create a log file, indulge it. It’s about to pull you out of a hole.

STEP 4

Now, at this point, if you’re lucky, you should see your drive. And you can proceed to the last steps. But let’s assume you’re not, that you have, say, a multi-boot machine. In this case, ownerships can get blurry, and Testdisk needs your permission to open them. You’ll see something like this:

How to recover deleted files in Linux using TestDisk
Sometimes you may need sudo rights

Select “sudo” and enter your password. Hit “enter” and “enter” again on the next screen to create another log file.

STEP 5

This time Testdisk displays all your drives. Arrow key to the drive in question and hit enter.

How to recover deleted files in Linux using TestDisk
You’ll have to select the drive where you are looking for files

STEP 6

Testdisk has again selected the correct setting. This makes sense since a simple storage device is seldom partitioned. Again hit enter:

How to recover deleted files in Linux using TestDisk

STEP 7

And finally we have to do a little thinking to do. If you read the first screen—and I’ll bet you didn’t—this program isn’t just for recovering deleted files. It’s a powerful disk utility. But if we remember what we’re trying to do the choice is fairly obvious: we’re not trying to fix a disk, we’re trying to recover a file. Select “Advanced” and hit “enter”.

How to recover deleted files in Linux using TestDisk
Select Advanced

STEP 8

At the bottom of the page choose “Undelete” and get ready to see a ghost!

How to recover deleted files in Linux using TestDisk
Select Undelete

STEP 9

Testdisk will scan for files and produce a list of deleted files highlighted in red. Arrow down to it and carefully read the choices at the bottom.

How to recover deleted files in Linux using TestDisk

STEP 10

Again, bear in mind that Testdisk is a multi-function tool. Most of these options deal with groups of files; we only want our damn resume back! So hit “c”.

How to recover deleted files in Linux using TestDisk
Hit C to copy and thus recover the deleted file

As you can see from the scoreboard, we’ve won 1-0. After hitting “c” there are options about where you might want to recover the file to, but it defaults to your home folder. And again this is generally the best thing to do. Navigating in Testdisk is a little tricky, whereas dragging and dropping after the fact is a breeze.

A FEW TIPS ON RECOVERING DELETED FILES IN LINUX USING TESTDISK

First, if you find yourself somewhere you don’t want to be, hit “q” for quit. This won’t close the program, instead, it will act like the “back” button on a program with a full blown GUI, and put you back a page. And just like a “back” button repeating will eventually lead you back to the beginning.

Second, as with anything, the fewer the distractions, the easier it is to find what you’re looking for. In other words, physically detach all other storage drives. In graphically simple environments simplicity is your friend.

Finally, Testdisk can also help you retrieve files that have become inaccessible for other reasons. In fact, this is why I started using the program in the first place. I was trying to save files from a corrupted drive that could not be made to boot. Normally it’s simply a matter of removing said drive any hooking it up to a USB adapter. You can then mount it on another PC and copy the files where ever you want.

But what if the drive is formatted to LVM? This was my problem because a mounted LVM drive looks nothing like a normal Linux OS. None of the usual files appear, and hunting around simply doesn’t help. This, among other reasons, is because most Linux file managers can no longer read ext.2 file systems.

Nevertheless, after a few false starts, I was able to find and save the missing files. Note, however, that the sequence of steps here will be a little different, you may need to use the “analyze” option for Testdisk to make sense of the drive and you may have to poke around a little to find the “home” folder once you do. Furthermore, the files you’re looking for will not appear in red since they were never deleted in the first place. But once you do find them, the copying procedure is basically the same.

With Testdisk and a little luck, you may never lose your resume again as you can always recover deleted files in Linux.

Compatibility with Older Systems RHEL 7

If an ACL has been set on any file on a given file system, that file system has the ext_attr attribute. This attribute can be seen using the following command:
# tune2fs -l filesystem-device
A file system that has acquired the ext_attr attribute can be mounted with older kernels, but those kernels do not enforce any ACLs which have been set.
Versions of the e2fsck utility included in version 1.22 and higher of the e2fsprogs package (including the versions in Red Hat Enterprise Linux 2.1 and 4) can check a file system with the ext_attr attribute. Older versions refuse to check it.

Retrieving Acls RHEL 7

To determine the existing ACLs for a file or directory, use the getfacl command. In the example below, the getfacl is used to determine the existing ACLs for a file.

Example 4.4. Retrieving ACLs

# getfacl home/john/picture.png
The above command returns the following output:
# file: home/john/picture.png 
# owner: john 
# group: john 
user::rw- 
group::r-- 
other::r--
If a directory with a default ACL is specified, the default ACL is also displayed as illustrated below. For example, getfacl home/sales/ will display similar output:
# file: home/sales/ 
# owner: john 
# group: john 
user::rw- 
user:barryg:r-- 
group::r-- 
mask::r-- 
other::r-- 
default:user::rwx 
default:user:john:rwx 
default:group::r-x 
default:mask::rwx 
default:other::r-x

 

Setting Access Acls RHEL 7

There are two types of ACLs: access ACLs and default ACLs. An access ACL is the access control list for a specific file or directory. A default ACL can only be associated with a directory; if a file within the directory does not have an access ACL, it uses the rules of the default ACL for the directory. Default ACLs are optional.
ACLs can be configured:
  1. Per user
  2. Per group
  3. Via the effective rights mask
  4. For users not in the user group for the file
The setfacl utility sets ACLs for files and directories. Use the -m option to add or modify the ACL of a file or directory:
# setfacl -m rules files
Rules (rules) must be specified in the following formats. Multiple rules can be specified in the same command if they are separated by commas.
u:uid:perms
Sets the access ACL for a user. The user name or UID may be specified. The user may be any valid user on the system.
g:gid:perms
Sets the access ACL for a group. The group name or GID may be specified. The group may be any valid group on the system.
m:perms
Sets the effective rights mask. The mask is the union of all permissions of the owning group and all of the user and group entries.
o:perms
Sets the access ACL for users other than the ones in the group for the file.
Permissions (perms) must be a combination of the characters r, w, and x for read, write, and execute.
If a file or directory already has an ACL, and the setfacl command is used, the additional rules are added to the existing ACL or the existing rule is modified.

Example 4.1. Give read and write permissions

For example, to give read and write permissions to user andrius:
# setfacl -m u:andrius:rw /project/somefile
To remove all the permissions for a user, group, or others, use the -x option and do not specify any permissions:
# setfacl -x rules files

Example 4.2. Remove all permissions

For example, to remove all permissions from the user with UID 500:
# setfacl -x u:500 /project/somefile

 

Using Command-line Tools to manage users RHEL 7

Apart from the Users settings tool described in Section 3.2, “Managing Users in a Graphical Environment”, which is designed for basic managing of users, you can use command line tools for managing users and groups that are listed in Table 3.1, “Command line utilities for managing users and groups”.

Table 3.1. Command line utilities for managing users and groups

Utilities Description
id Displays user and group IDs.
useradd, usermod, userdel Standard utilities for adding, modifying, and deleting user accounts.
groupadd, groupmod, groupdel Standard utilities for adding, modifying, and deleting groups.
gpasswd Utility primarily used for modification of group password in the /etc/gshadow file which is used by the newgrp command.
pwck, grpck Utilities that can be used for verification of the password, group, and associated shadow files.
pwconv, pwunconv Utilities that can be used for the conversion of passwords to shadow passwords, or back from shadow passwords to standard passwords.
grpconv, grpunconv Similar to the previous, these utilities can be used for conversion of shadowed information for group accounts.

3.3.1. Adding a New User

To add a new user to the system, type the following at a shell prompt as root:
useradd [options] username
…where options are command-line options as described in Table 3.2, “Common useradd command-line options”.
By default, the useradd command creates a locked user account. To unlock the account, run the following command as root to assign a password:
passwd username
Optionally, you can set a password aging policy.

Table 3.2. Common useradd command-line options

Option Description
-c ‘comment comment can be replaced with any string. This option is generally used to specify the full name of a user.
-d home_directory Home directory to be used instead of default /home/username/.
-e date Date for the account to be disabled in the format YYYY-MM-DD.
-f days Number of days after the password expires until the account is disabled. If 0 is specified, the account is disabled immediately after the password expires. If -1 is specified, the account is not disabled after the password expires.
-g group_name Group name or group number for the user’s default (primary) group. The group must exist prior to being specified here.
-G group_list List of additional (supplementary, other than default) group names or group numbers, separated by commas, of which the user is a member. The groups must exist prior to being specified here.
-m Create the home directory if it does not exist.
-M Do not create the home directory.
-N Do not create a user private group for the user.
-p password The password encrypted with crypt.
-r Create a system account with a UID less than 1000 and without a home directory.
-s User’s login shell, which defaults to /bin/bash.
-u uid User ID for the user, which must be unique and greater than 999.
The command-line options associated with the usermod command are essentially the same. Note that if you want to add a user to another supplementary group, you need to use the -a, --append option with the -G option. Otherwise the list of supplementary groups for the user will be overwritten by those specified with the usermod -G command.

 

IMPORTANT

The default range of IDs for system and normal users has been changed in Red Hat Enterprise Linux 7 from earlier releases. Previously, UID 1-499 was used for system users and values above for normal users. The default range for system users is now 1-999. This change might cause problems when migrating to Red Hat Enterprise Linux 7 with existing users having UIDs and GIDs between 500 and 999. The default ranges of UID and GID can be changed in the /etc/login.defs file.

Explaining the Process

The following steps illustrate what happens if the command useradd juan is issued on a system that has shadow passwords enabled:
  1. A new line for juan is created in /etc/passwd:
    juan:x:1001:1001::/home/juan:/bin/bash
    The line has the following characteristics:
    • It begins with the user name juan.
    • There is an x for the password field indicating that the system is using shadow passwords.
    • A UID greater than 999 is created. Under Red Hat Enterprise Linux 7, UIDs below 1000 are reserved for system use and should not be assigned to users.
    • A GID greater than 999 is created. Under Red Hat Enterprise Linux 7, GIDs below 1000 are reserved for system use and should not be assigned to users.
    • The optional GECOS information is left blank. The GECOS field can be used to provide additional information about the user, such as their full name or phone number.
    • The home directory for juan is set to /home/juan/.
    • The default shell is set to /bin/bash.
  2. A new line for juan is created in /etc/shadow:
    juan:!!:14798:0:99999:7:::
    The line has the following characteristics:
    • It begins with the username juan.
    • Two exclamation marks (!!) appear in the password field of the /etc/shadow file, which locks the account.

      NOTE

      If an encrypted password is passed using the -p flag, it is placed in the /etc/shadow file on the new line for the user.
    • The password is set to never expire.
  3. A new line for a group named juan is created in /etc/group:
    juan:x:1001:
    A group with the same name as a user is called a user private group. For more information on user private groups.
    The line created in /etc/group has the following characteristics:
    • It begins with the group name juan.
    • An x appears in the password field indicating that the system is using shadow group passwords.
    • The GID matches the one listed for juan‘s primary group in /etc/passwd.
  4. A new line for a group named juan is created in /etc/gshadow:
    juan:!::
    The line has the following characteristics:
    • It begins with the group name juan.
    • An exclamation mark (!) appears in the password field of the /etc/gshadow file, which locks the group.
    • All other fields are blank.
  5. A directory for user juan is created in the /home directory:
    ~]# ls -ld /home/juan
    drwx------. 4 juan juan 4096 Mar  3 18:23 /home/juan
    This directory is owned by user juan and group juan. It has read, write, and execute privileges only for the user juan. All other permissions are denied.
  6. The files within the /etc/skel/ directory (which contain default user settings) are copied into the new /home/juan/ directory:
    ~]# ls -la /home/juan
    total 28
    drwx------. 4 juan juan 4096 Mar  3 18:23 .
    drwxr-xr-x. 5 root root 4096 Mar  3 18:23 ..
    -rw-r--r--. 1 juan juan   18 Jun 22  2010 .bash_logout
    -rw-r--r--. 1 juan juan  176 Jun 22  2010 .bash_profile
    -rw-r--r--. 1 juan juan  124 Jun 22  2010 .bashrc
    drwxr-xr-x. 4 juan juan 4096 Nov 23 15:09 .mozilla
At this point, a locked account called juan exists on the system. To activate it, the administrator must next assign a password to the account using the passwd command and, optionally, set password aging guidelines.

3.3.2. Adding a New Group

To add a new group to the system, type the following at a shell prompt as root:
groupadd [options] group_name
…where options are command-line options as described in Table 3.3, “Common groupadd command-line options”.

Table 3.3. Common groupadd command-line options

Option Description
-f, --force When used with -g gid and gid already exists, groupadd will choose another unique gid for the group.
-g gid Group ID for the group, which must be unique and greater than 999.
-K, --key key=value Override /etc/login.defs defaults.
-o, --non-unique Allows creating groups with duplicate GID.
-p, --password password Use this encrypted password for the new group.
-r Create a system group with a GID less than 1000.

3.3.3. Creating Group Directories

System administrators usually like to create a group for each major project and assign people to the group when they need to access that project’s files. With this traditional scheme, file management is difficult; when someone creates a file, it is associated with the primary group to which they belong. When a single person works on multiple projects, it becomes difficult to associate the right files with the right group. However, with the UPG scheme, groups are automatically assigned to files created within a directory with the setgid bit set. The setgid bit makes managing group projects that share a common directory very simple because any files a user creates within the directory are owned by the group that owns the directory.
For example, a group of people need to work on files in the /opt/myproject/ directory. Some people are trusted to modify the contents of this directory, but not everyone.
  1. As root, create the /opt/myproject/ directory by typing the following at a shell prompt:
    mkdir /opt/myproject
  2. Add the myproject group to the system:
    groupadd myproject
  3. Associate the contents of the /opt/myproject/ directory with the myproject group:
    chown root:myproject /opt/myproject
  4. Allow users in the group to create files within the directory and set the setgid bit:
    chmod 2775 /opt/myproject
    At this point, all members of the myproject group can create and edit files in the /opt/myproject/ directory without the administrator having to change file permissions every time users write new files. To verify that the permissions have been set correctly, run the following command:
    ~]# ls -ld /opt/myproject
    drwxrwsr-x. 3 root myproject 4096 Mar  3 18:31 /opt/myproject
    
  5. Add users to the myproject group:
    usermod -aG myproject username

3.3.4. Setting Default Permissions for New Files Using umask

When a process creates a file, the file has certain default permissions, for example, -rw-rw-r--. These initial permissions are partially defined by the file mode creation mask, also called file permission mask or umask. Every process has its own umask, for example, bash has umask 0022 by default. Process umask can be changed.

⁠What umask consists of

A umask consists of bits corresponding to standard file permissions. For example, for umask 0137, the digits mean that:
  • 0 = no meaning, it is always 0 (umask does not affect special bits)
  • 1 = for owner permissions, the execute bit is set
  • 3 = for group permissions, the execute and write bits are set
  • 7 = for others permissions, the execute, write, and read bits are set
Umasks can be represented in binary, octal, or symbolic notation. For example, the octal representation 0137 equals symbolic representation u=rw-,g=r--,o=---. Symbolic notation specification is the reverse of the octal notation specification: it shows the allowed permissions, not the prohibited permissions.

⁠How umask works

Umask prohibits permissions from being set for a file:
  • When a bit is set in umask, it is unset in the file.
  • When a bit is not set in umask, it can be set in the file, depending on other factors.
The following figure shows how umask 0137 affects creating a new file.

Applying umask when creating a file

Figure 3.3. Applying umask when creating a file

IMPORTANT

For security reasons, a regular file cannot have execute permissions by default. Therefore, even if umask is 0000, which does not prohibit any permissions, a new regular file still does not have execute permissions. However, directories can be created with execute permissions:
[john@server tmp]$ umask 0000
[john@server tmp]$ touch file
[john@server tmp]$ mkdir directory
[john@server tmp]$ ls -lh .
total 0
drwxrwxrwx. 2 john john 40 Nov  2 13:17 directory
-rw-rw-rw-. 1 john john  0 Nov  2 13:17 file

3.3.4.1. Managing umask in Shells

For popular shells, such as bash, ksh, zsh and tcsh, umask is managed using the umask shell builtin. Processes started from shell inherit its umask.

⁠Displaying the current mask

To show the current umask in octal notation:
~]$ umask
0022
To show the current umask in symbolic notation:
~]$ umask -S
u=rwx,g=rx,o=rx

⁠Setting mask in shell using umask

To set umask for the current shell session using octal notation run:
~]$ umask octal_mask
Substitute octal_mask with four or less digits from 0 to 7. When three or less digits are provided, permissions are set as if the command contained leading zeros. For example, umask 7 translates to 0007.

Example 3.1. Setting umask Using Octal Notation

To prohibit new files from having write and execute permissions for owner and group, and from having any permissions for others:
~]$ umask 0337
Or simply:
~]$ umask 337
To set umask for the current shell session using symbolic notation:
~]$ umask -S symbolic_mask

Example 3.2. Setting umask Using Symbolic Notation

To set umask 0337 using symbolic notation:
~]$ umask -S u=r,g=r,o=

⁠Working with the default shell umask

Shells usually have a configuration file where their default umask is set. For bash, it is /etc/bashrc. To show the default bash umask:
~]$ grep -i -B 1 umask /etc/bashrc
The output shows if umask is set, either using the umask command or the UMASK variable. In the following example, umask is set to 022 using the umask command:
~]$ grep -i -B 1 umask /etc/bashrc
    # By default, we want umask to get set. This sets it for non-login shell.
--
    if [ $UID -gt 199 ] && [ "`id -gn`" = "`id -un`" ]; then
       umask 002
    else
       umask 022
To change the default umask for bash, change the umask command call or the UMASK variable assignment in /etc/bashrc. This example changes the default umask to 0227:
    if [ $UID -gt 199 ] && [ "`id -gn`" = "`id -un`" ]; then
       umask 002
    else
       umask 227

⁠Working with the default shell umask of a specific user

By default, bash umask of a new user defaults to the one defined in /etc/bashrc.
To change bash umask for a particular user, add a call to the umask command in $HOME/.bashrc file of that user. For example, to change bash umask of user john to 0227:
john@server ~]$ echo 'umask 227' >> /home/john/.bashrc

⁠Setting default permissions for newly created home directories

To change permissions with which user home directories are created, change the UMASK variable in the /etc/login.defs file:

# The permission mask is initialized to this value. If not specified,
# the permission mask will be initialized to 022.
UMASK 077

 

Managing Users in a Graphical Environment RHEL 7

The Users utility allows you to view, modify, add, and delete local users in the graphical user interface.

3.2.1. Using the Users Settings Tool

Press the Super key to enter the Activities Overview, type Users and then press Enter. The Users settings tool appears. The Super key appears in a variety of guises, depending on the keyboard and other hardware, but often as either the Windows or Command key, and typically to the left of the Spacebar. Alternatively, you can open the Users utility from the Settings menu after clicking your user name in the top right corner of the screen.
To make changes to the user accounts, first select the Unlock button and authenticate yourself as indicated by the dialog box that appears. Note that unless you have superuser privileges, the application will prompt you to authenticate as root. To add and remove users, select the + and button respectively. To add a user to the administrative group wheel, change the Account Type from Standard to Administrator. To edit a user’s language setting, select the language and a drop-down menu appears.
The Users Settings Tool

Figure 3.1. The Users Settings Tool

When a new user is created, the account is disabled until a password is set. The Password drop-down menu, shown in Figure 3.2, “The Password Menu”, contains the options to set a password by the administrator immediately, choose a password by the user at the first login, or create a guest account with no password required to log in. You can also disable or enable an account from this menu.
The Password Menu

Figure 3.2. The Password Menu

Overview of File System Hierarchy Standard (FHS)

Red Hat Enterprise Linux uses the Filesystem Hierarchy Standard (FHS) file system structure, which defines the names, locations, and permissions for many file types and directories.
The FHS document is the authoritative reference to any FHS-compliant file system, but the standard leaves many areas undefined or extensible. This section is an overview of the standard and a description of the parts of the file system not covered by the standard.
Compliance with the standard means many things, but the two most important are compatibility with other compliant systems and the ability to mount a /usr/ partition as read-only. This second point is important because the directory contains common executables and should not be changed by users. Also, since the /usr/ directory is mounted as read-only, it can be mounted from the CD-ROM or from another machine via a read-only NFS mount.

⁠1.2.1. FHS Organization

The directories and files noted here are a small subset of those specified by the FHS document. Refer to the latest FHS document for the most complete information.
The complete standard is available online at http://www.pathname.com/fhs/.

⁠1.2.1.1. The /boot/ Directory

The /boot/ directory contains static files required to boot the system, such as the Linux kernel. These files are essential for the system to boot properly.

Warning

Do not remove the /boot/ directory. Doing so renders the system unbootable.

⁠1.2.1.2. The /dev/ Directory

The /dev/ directory contains device nodes that either represent devices that are attached to the system or virtual devices that are provided by the kernel. These device nodes are essential for the system to function properly. The udev daemon takes care of creating and removing all these device nodes in /dev/.
Devices in the /dev directory and subdirectories are either character (providing only a serial stream of input/output) or block (accessible randomly). Character devices include mouse, keyboard, modem while block devices include hard disk, floppy drive etc. If you have GNOME or KDE installed in your system, devices such as external drives or cds are automatically detected when connected (e.g via usb) or inserted (e.g via CD or DVD drive) and a popup window displaying the contents is automatically displayed. Files in the /dev directory are essential for the system to function properly.

Table 1.1. Examples of common files in the /dev

File Description
/dev/hda The master device on primary IDE channel.
/dev/hdb The slave device on primary IDE channel.
/dev/tty0 The first virtual console.
/dev/tty1 The second virtual console.
/dev/sda The first device on primary SCSI or SATA channel.
/dev/lp0 The first parallel port.

⁠1.2.1.3. The /etc/ Directory

The /etc/ directory is reserved for configuration files that are local to the machine. No binaries are to be placed in /etc/. Any binaries that were once located in /etc/ should be placed into /sbin/ or /bin/.
Examples of directories in /etc are the X11/ and skel/:
/etc
   |- X11/
   |- skel/
The /etc/X11/ directory is for X Window System configuration files, such as xorg.conf. The /etc/skel/ directory is for “skeleton” user files, which are used to populate a home directory when a user is first created. Applications also store their configuration files in this directory and may reference them when they are executed.

⁠1.2.1.4. The /lib/ Directory

The /lib/ directory should contain only those libraries needed to execute the binaries in /bin/ and /sbin/. These shared library images are particularly important for booting the system and executing commands within the root file system.

⁠1.2.1.5. The /media/ Directory

The /media/ directory contains subdirectories used as mount points for removable media such as usb storage media, DVDs, CD-ROMs, and Zip disks.

⁠1.2.1.6. The /mnt/ Directory

The /mnt/ directory is reserved for temporarily mounted file systems, such as NFS file system mounts. For all removable media, please use the /media/ directory. Automatically detected removable media will be mounted in the /media directory.

Note

The /mnt directory must not be used by installation programs.

⁠1.2.1.7. The /opt/ Directory

The /opt/ directory provides storage for most application software packages.
A package placing files in the /opt/ directory creates a directory bearing the same name as the package. This directory, in turn, holds files that otherwise would be scattered throughout the file system, giving the system administrator an easy way to determine the role of each file within a particular package.
For example, if sample is the name of a particular software package located within the /opt/ directory, then all of its files are placed in directories inside the /opt/sample/ directory, such as /opt/sample/bin/ for binaries and /opt/sample/man/ for manual pages.
Packages that encompass many different sub-packages, data files, extra fonts, clipart etc are also located in the /opt/ directory, giving that large package a way to organize itself. In this way, our sample package may have different tools that each go in their own sub-directories, such as /opt/sample/tool1/ and /opt/sample/tool2/, each of which can have their own bin/, man/, and other similar directories.

⁠1.2.1.8. The /proc/ Directory

The /proc/ directory contains special files that either extract information from or send information to the kernel. Examples include system memory, cpu information, hardware configuration etc.
Due to the great variety of data available within /proc/ and the many ways this directory can be used to communicate with the kernel, an entire chapter has been devoted to the subject.

⁠1.2.1.9. The /sbin/ Directory

The /sbin/ directory stores executables used by the root user. The executables in /sbin/ are used at boot time, for system administration and to perform system recovery operations. Of this directory, the FHS says:

/sbin contains binaries essential for booting, restoring, recovering, and/or repairing the system in addition to the binaries in /bin. Programs executed after /usr/ is known to be mounted (when there are no problems) are generally placed into /usr/sbin. Locally-installed system administration programs should be placed into /usr/local/sbin.
At a minimum, the following programs should be in /sbin/:
arp, clock,
halt, init,
fsck.*, grub,
ifconfig, mingetty,
mkfs.*, mkswap,
reboot, route,
shutdown, swapoff,
swapon

⁠1.2.1.10. The /srv/ Directory

The /srv/ directory contains site-specific data served by your system running Red Hat Enterprise Linux. This directory gives users the location of data files for a particular service, such as FTP, WWW, or CVS. Data that only pertains to a specific user should go in the /home/ directory.

⁠1.2.1.11. The /sys/ Directory

The /sys/ directory utilizes the new sysfs virtual file system specific to the 2.6 kernel. With the increased support for hot plug hardware devices in the 2.6 kernel, the /sys/ directory contains information similarly held in /proc/, but displays a hierarchical view of specific device information in regards to hot plug devices.

⁠1.2.1.12. The /usr/ Directory

The /usr/ directory is for files that can be shared across multiple machines. The /usr/ directory is often on its own partition and is mounted read-only. At a minimum, the following directories should be subdirectories of /usr/:
/usr
   |- bin/
   |- etc/
   |- games/
   |- include/
   |- kerberos/
   |- lib/
   |- libexec/
   |- local/
   |- sbin/
   |- share/
   |- src/
   |- tmp -> ../var/tmp/
Under the /usr/ directory, the bin/ subdirectory contains executables, etc/ contains system-wide configuration files, games is for games, include/ contains C header files, kerberos/ contains binaries and other Kerberos-related files, and lib/ contains object files and libraries that are not designed to be directly utilized by users or shell scripts. The libexec/ directory contains small helper programs called by other programs, sbin/ is for system administration binaries (those that do not belong in the /sbin/ directory), share/ contains files that are not architecture-specific, src/ is for source code.

⁠1.2.1.13. The /usr/local/ Directory

The FHS says:

The /usr/local hierarchy is for use by the system administrator when installing software locally. It needs to be safe from being overwritten when the system software is updated. It may be used for programs and data that are shareable among a group of hosts, but not found in /usr.
The /usr/local/ directory is similar in structure to the /usr/ directory. It has the following subdirectories, which are similar in purpose to those in the /usr/ directory:
/usr/local
	|- bin/
	|- etc/
	|- games/
	|- include/
	|- lib/
	|- libexec/
	|- sbin/
	|- share/
	|- src/
In Red Hat Enterprise Linux, the intended use for the /usr/local/ directory is slightly different from that specified by the FHS. The FHS says that /usr/local/ should be where software that is to remain safe from system software upgrades is stored. Since software upgrades can be performed safely with RPM Package Manager (RPM), it is not necessary to protect files by putting them in /usr/local/. Instead, the /usr/local/directory is used for software that is local to the machine.
For instance, if the /usr/ directory is mounted as a read-only NFS share from a remote host, it is still possible to install a package or program under the /usr/local/ directory.

⁠1.2.1.14. The /var/ Directory

Since the FHS requires Linux to mount /usr/ as read-only, any programs that write log files or need spool/ or lock/ directories should write them to the /var/ directory. The FHS states /var/ is for:

…variable data files. This includes spool directories and files, administrative and logging data, and transient and temporary files.
Below are some of the directories found within the /var/ directory:
/var
   |- account/
   |- arpwatch/
   |- cache/
   |- crash/
   |- db/
   |- empty/
   |- ftp/
   |- gdm/
   |- kerberos/
   |- lib/
   |- local/
   |- lock/
   |- log/
   |- mail -> spool/mail/
   |- mailman/
   |- named/
   |- nis/
   |- opt/
   |- preserve/
   |- run/
   +- spool/
       |- at/
       |- clientmqueue/
       |- cron/
       |- cups/
       |- exim/
       |- lpd/
       |- mail/
       |- mailman/
       |- mqueue/
       |- news/
       |- postfix/
       |- repackage/
       |- rwho/
       |- samba/
       |- squid/
       |- squirrelmail/
       |- up2date/
       |- uucp
       |- uucppublic/
       |- vbox/
|- tmp/
|- tux/
|- www/
|- yp/
System log files, such as messages and lastlog, go in the /var/log/ directory. The /var/lib/rpm/ directory contains RPM system databases. Lock files go in the /var/lock/ directory, usually in directories for the program using the file. The /var/spool/ directory has subdirectories for programs in which data files are stored.