Linux · Raspberry Pi

Use USB hard disk & flash drives with your Raspberry Pi


Updated: November 2017 to change reference of OS X to macOS and mention the APFS.

This article was meant to be a brief piece on how to mount an external USB drive, but it quickly spiralled out when I started writing about all the nuances and potential issues one might encounter. So I have a created a quick summary of the commands on how to mount a drive below. But I highly recommend reading the rest of the article as there are a number of potential pitfalls with the Pi and external USB drives that are addressed.

Brief

To mount a USB drive:

sudo mkdir /mnt/usbdrive
sudo mount /dev/sda1 /mnt/usbdrive
ls /mnt/usbdrive

To list your file systems:

sudo fdisk -l
sudo mount -l
df -h

Before disconnecting a USB drive:

sudo umount /dev/sda1

Format a drive to EXT4

sudo mkfs.ext4 /dev/sda1 -L untitled

Add macOS HFS+ read/write support

sudo apt-get install hfsutils hfsprogs hfsutils

Format a drive to HFS+

sudo mkfs.hfsplus /dev/sda1 -v untitled

Add Windows NTFS read/write support

sudo apt-get install ntfs-3g

Format a drive to NTFS

sudo mkfs.ntfs /dev/sda1 -f -v -I -L untitled

Add Windows/DOS FAT32 read/write support

sudo apt-get install dosfstools

Format a drive to FAT32

sudo mkfs.vfat /dev/sda1 -n untitled

In summary

The Raspberry Pi is a great and flexible little device. But one of its limitations is the storage options if you wish to use the machine as a file or multimedia server. SD memory cards are cheap and popular with low-end specifications but reach an affordability and storage cap when their sizes increase. A cheap USB powered external drive with many times more space can be had for a similar price to a top capacity SD memory card.

USB power problems

A significant limitation for running a USB drive on a Raspberry Pi is the power requirements. The Universal Serial Bus specification states that to adhere to the standard up to 0.5A (amps) can be drawn from a single port.

My old Samsung G2 Portable 640 hard drive requires 0.85A to work which is +0.35A above the USB2 specification. The drive is not an isolated example. Many modern desktop PCs and laptops supply a higher amperage than the 0.5A specification to their USB ports to support devices such as portable hard drives. Unfortunately, the Pi cannot power many external drives.

To get around this problem you need a powered USB hub. Attach the hub device to the Pi’s USB port, plug the USB hard drive into one of the hub’s ports and insert the hub’s power supply into a walled power socket. My little Logitech ‘Premium 4-port’ USB hub can share 2.5A between 4 devices which is more than enough to power my Samsung (0.85A) drive.

I would also recommend against powering the Raspberry Pi off the same USB Hub as a USB drive. For me, this caused interference where the hard drive would momentarily lose power.

Mounting a drive

When a hard drive is mounted, it connects to your Pi. Linux recognises it and assigns a directory where you can access its content. These directories are known as mount points and can be given any name that works for you but they should be placed in /mnt.

I will call my mount point ‘usbdrive‘. First, we need to create a mount point.

sudo mkdir /mnt/usbdrive

Linux has the /dev directory that is in use to store special files that provide access to the computer hardware. The /dev/sd* collection of files where each letter represents a drive connected to your Raspberry Pi.

/dev/sda Would be your first connected drive.
/dev/sdb Would be your second drive.

To mount the drive to your mount point ‘usbdrive‘.

sudo mount /dev/sda1 /mnt/usbdrive

The numeric 1 at the end of /dev/sda is a requirement and tells Linux to mount the first partition.

Partitions are beyond the scope of this article, but you can learn more about that at the Ubuntu Community Docs.

mkdir mnt usbdrive

Disconnect / unmount a drive

It is always advisable that you unmount a USB drive before unplugging it from its power source. This forces all queued data to be written to the drive before it loses power.

sudo umount /dev/sda1

You may need to use the -f force option if the drive does not dismount.

sudo umount -f /dev/sda1

If you use the shutdown -P -h 0 command to power down your Pi you do not need to unmount.

Disk file systems

The disk file system is a method an operating system stores and reads data on a drive. There is an endless number of disk file systems out there as each operating system seems to have their native but incompatible system.

Linux has many native file formats but today the most common is the EXT (Extended File System) set which includes ext2, ext3 and ext4.

Apple’s macOS used to run HFS+ (Hierarchical File System Plus) otherwise known as Mac OS Extended but in 2017 switched to APFS (Apple File System).

Modern Microsoft Windows systems mostly use NTFS (New Technology File System).

Legacy Microsoft Windows systems and ancient Microsoft DOS systems use a variation of the FAT (File Allocation Table) which includes FAT, VFAT, FAT32 and exFAT.

High CPU usage with the ntfs-3 driver
Excessive CPU usage with the ntfs-3 driver that slows Samba (smdb) transfers
xxx
With EXT4 the file transfer using Samba has an additional 250%+ CPU resource available for use

Disk file systems compatibility

EXT has native support within Linux and the Raspberry Pi. It has no official support in Windows. There are free third-party drivers available for Windows offering limited read/write EXT support such as the open source EXT2FSD or EXT2Read. macOS users need to use the commercial Paragon ExtFS to enable full EXT support.

To enable Linux EXT4 support:
It is turned on by default on the Raspberry Pi.

HFS+ has restricted support in Linux. It can read HFS+ formatted drives but can only write to them if journaling is disabled. Windows has no native HFS+ support, but there are commercial solutions such as Paragon HFS+ for Windows.

To enable Linux HFS+ support:
sudo apt-get install hfsutils hfsprogs hfsutils

FAT is probably the most supported file system, but it is also the most limited. Linux, Windows and macOS all support FAT, VFAT and FAT32. ExFAT otherwise known as FAT64 is native to modern Windows and macOS but has no support in Linux due to patient incompatibilities.

To enable Linux FAT32 support:
sudo apt-get install dosfstools

NTFS has read-only support for Linux and macOS. Third party drivers are available to add write support including the commercial Paragon NTFS and the open source NTFS-3g.

To enable Linux NTFS support:
sudo apt-get install ntfs-3g

Performance issues & which disk file system to use?

As a non-scientific test, I took a 4GB video file and copied it to various file systems using my Raspberry Pi and the USB hard drive.

The worst performer by far was the NTFS-3g driver for Linux NTFS read and write support.

The transfer that 4GB file from my Windows 7 PC to the NTFS formatted USB hard drive took around a minute or two. The same data from the Raspberry Pi’s SD memory card to the NTFS formatted USB drive took 30 minutes to write and 23 minutes to read!

Performance for EXT3, EXT4 and FAT32 were about the same at 12-14 minutes to both read and write. It suggests that there is a bottleneck with either the SD memory card or USB drivers and not the file system.

If the Pi regularly uses the drive, my recommendation would be to use EXT4. EXT4 is often found in the Linux world and is principally the same as EXT3 with some more minor features.

FAT32 is the most compatible file system but has a restrictive 4GB file size limit.

EXT2, HFS+ on Linux and FAT32 lack journaling support. It means these file systems can’t elegantly recover if they lose power and makes them prone to errors when used on portable drives.

FAT32, NTFS can not store Linux file or user permissions.

Format a drive

To change the file system to a drive you need to format it. Linux allows you to format any supported disk format using the mkfs tool.

In the examples below, you will notice an option followed by ‘untitled‘. These are optional volume labels to name your drive.

First, you must unmount the drive you wish to format.

sudo umount /dev/sda1

To format a drive to EXT3 (Linux):
sudo mkfs.ext3 /dev/sda1 -L untitled

To format a drive to EXT4 (Linux):
sudo mkfs.ext4 /dev/sda1 -L untitled

To format a drive to HFS+ (Mac OS X):
sudo mkfs.hfsplus /dev/sda1 -v untitled

To format a drive to FAT32 (DOS and legacy Windows):
sudo mkfs.vfat /dev/sda1 -n untitled

To format a drive to NTFS (Windows):
sudo mkfs.ntfs /dev/sda1 -f -v -I -L untitled

I have applied a few options here that I will explain.
-f Fast Format. Due to the poor performance of 3g.ntfs on the Pi I highly recommend using the less CPU intensive fast format mode.
-v Verbose. By default the NTFS status output is limited, so this lets you know what is happening.
-I Disable Windows Indexing. It improves the write performance of the drive, but Windows Search queries will take longer.

Format NTFS
Format NTFS
Format HFS+
Format HFS+
Format FAT32 (vfat)
Format FAT32 (vfat)
Format EXT4
Format EXT4

Automatically mount a drive

To simplify the process of mounting a drive, you can add information to the fstab settings file located in /etc/. I would recommend taking a look at the Ubuntu FSTAB community page for a deeper understanding of this.

First, run nano to edit fstab. The -Bw options tell nano to backup the file and not to use any line-wrap.

sudo nano -Bw /etc/fstab

You should already see some existing entries. Do NOT change these as the two /mnt/mmcblk0p entries are there to mount the SD card.

Add the following to the bottom of the file.

/dev/sda1 /mnt/usbdisk auto defaults,user 0 1

Explanation:

/dev/sda1 Is the location of the drive to mount.
/mnt/usbdisk Is the mount point, which is the folder to access the content of the drive.
auto Is the file system type, and here you can set ‘auto‘ or force a file system type such as ext2, ext3, ext4, hfsplus, ntfs, vfat.
defaults,user Are mount options. You normally only need to supply ‘defaults‘. Though some options might be useful, ‘ro‘ for read-only or ‘user‘ to enable write permission for all users. Use a non-spaced comma to separate multiple options.
0 A binary value used for debugging. It is best to keep this set at zero.
1 Pass number for a file system check at boot. ‘0‘ (zero) to disable or ‘2‘ to enable.

Save the changes to fstab.

[Ctrl] x
Y at the Save modified buffer prompt.
[Enter] for the File name to Write: /etc/fstab prompt.

nano etc fstab

The drive will mount at boot as long as it is attached to the Pi. ITo mount a drive after you have plugged it in use mount with the automatic option.

sudo mount -a

 

Advertisements

14 thoughts on “Use USB hard disk & flash drives with your Raspberry Pi

  1. i had just spent the longest time trying to write files to an EXT4 system USB drive that’s attached to my pi – and finally found a the answer! you need to allow permissions and the following will enable it for user ‘pi’

    sudo mount /dev/sda1 /mnt/usbdrive
    sudo chown pi:pi /mnt/usbdrive
    sudo chmod 777 /mnt/usbdrive

  2. Thanks for the great info, a good summary of how to mount and use various file systems.

    I have found that large files through NTFS can be accelerated by using -o noatime,big_writes -t ntfs-3g when mounted.

    What I am trying to find is a way to make the auto-mounting use this option by default. My need is to allow NTFS drives (one at random and not used before) to be connected and have the ‘noatime,big_writes’ option applied.
    Not sure if fstab can include an option for this to apply to all ntfs drives or if there is a preferred way to do it.. udevil or udisks perhaps?

    1. Hi Warren, I am really sorry for the late reply.

      fstab seems to be the most common way to auto-mount on Debian-based distros. It also allows you to supply “options” such as those you need.

      I’d suggest taking a look at this Ubuntu doc on AutomaticallyMountPartitions https://help.ubuntu.com/community/AutomaticallyMountPartitions under the ‘Editing Ubuntu’s filesystem’ table.

      The NTFS-3g manual has a list of options you can apply to the driver.
      http://www.tuxera.com/community/ntfs-3g-manual/

    1. Hi Damir, ntfs-3g is a resource hog. It’s an issue that isn’t such a big deal on a standard Linux box but is for a low performance device such as a Pi. Hence the recommendation that you avoid NTFS on the Pi unless it is needed.

  3. > ransfer using Samba has an additional 250%+ CPU r

    File transfer with samba period is more resource intensive and slower throughput than NFS. Why are we attempting samba?

    Add microsoft’s unix packages to windows to let them experience NFS simplicity, power, and speed

  4. I noticed a duplicate package name (both here and on the cheat sheet):

    sudo apt-get install hfsutils hfsprogs hfsutils

    It appears ‘hfsutils’ is both the first and third package name. Is that a mistake?

    1. Hi Dan thanks for pointing it out as I believe it is so. Unfortunately I don’t have access to a Pi atm to confirm that, but when I do I’ll correct the article.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s