How to FIX High disk I/O caused by UpdateDB

Since the latest system update, I’ve been finding that at about midnight everyday, my computer starts using 100% CPU and ~100% disk I/O during a period of time. Everything was freezed and I must take a break until it’s finished. After a quick search by glances, iostat and iotop, I found updatedb is the cause of this problem.

UpdateDB - How to solve high cpu usage and disk I/O activity
UpdateDB – How to solve high cpu usage and disk I/O activity

How to FIX High disk I/O caused by UpdateDB

updatedb is part of mlocate which invokes a database update each day. It is simply an indexing system to keep a database listing all the files on your server that you can do searches from using the locate command at the shell.

What is locate command?

locate is a common Unix tool for quickly finding files by name. It offers speed improvements over the find tool by searching a pre-constructed database file, rather than the filesystem directly.
Here’s a good comparison of them.
The downside of this approach is that changes made since the construction of the database file cannot be detected by locate.
This problem can be minimised by scheduled database updates.

So at its specified time, your computer will be slowing down with high CPU usage and large of disk I/O activity. Here are messages if you look into the system log with journalctl:

archviet 19 12:00:13 arch64 systemd[1]: Starting Update locate database...
archviet 19 12:00:25 arch64 systemd[1]: Started Update locate database.

💡 Tip: Another command that may be useful is to run$ journalctl -f at the time of the slow down. That will show output from your system log in real time. If the slow down is caused by some program going haywire you may find the log spewing information about that repeatedly there.

$ systemctl status updatedb.timer
* updatedb.timer - Daily locate database update
   Loaded: loaded (/usr/lib/systemd/system/updatedb.timer; static)
  Drop-In: /etc/systemd/system/updatedb.timer.d
           `-updatedb.timer.conf
   Active: active (waiting) since sam. 2019-04-19 10:57:37 CEST; 5h 16min ago

Now, let’s take a look into /usr/lib/systemd/system/updatedb.timer:

[Unit]
Description=Daily locate database update

[Timer]
OnCalendar=daily
AccuracySec=12h
Persistent=true

As you see, OnCalendar=daily mean it’s running at midnight everyday, you can set to another time by chech the [Timer] section as you want or make it run weekly.

If you like me rather use locate to find something, you will not see any benefit from the annoying process. In my case I’m quite happy with disable updatedb.timer and only update its database before you run it.

  1. Firstly, I disable updatedb.timer completely by mask it.
    $ sudo systemctl mask updatedb.timer
  2. Then I changed the locate command to up force update the database by add alias command into .bashrc
    alias locate="sudo updatedb; locate"

Why is /var/lib/mlocate/mlocate.db very big?

They are several reasons which increase the size of /var/lib/mlocate/ folder.

  • mlocate keeps the database /var/lib/mlocate/mlocate.db as an index to allow the system to find files. It indexes all the files on your system then its size will increasing fast.
  • Sometime, you were shutted off the computer abruptly because the heavy load of updatedb that freezed everything. The old mlocate files could be leftovers from interrupted indexing with previous updates.
  • updatedb indexed files in your cloud drive that was mounted into your system.
  • there are a LOT of paths in a .git folder.

If you have lots and lots of files on your machine, you may want to consider pruning some paths from the database. You can do this in /etc/updatedb.conf under PRUNEPATHS. The solution I used was:

  1. Edit the /etc/updatedb.conf file.
  2. Add the paths, file types, protocols, folder name (.git, .hg…) which are pruned from updatedb database. Below is an example:
    # This file sets variables that are used by updatedb.
    # For more info, see the updatedb.conf(5) manpage.
    
    # Filesystems that are pruned from updatedb database
    PRUNEFS="afs anon_inodefs auto autofs bdev binfmt binfmt_misc cgroup cifs coda configfs cramfs cpuset debugfs devfs devpts devtmpfs ecryptfs eventpollfs exofs futexfs ftpfs fuse fusectl gfs gfs2 hostfs hugetlbfs inotifyfs iso9660 jffs2 lustre misc mqueue ncpfs nnpfs ocfs ocfs2 pipefs proc ramfs rpc_pipefs securityfs selinuxfs sfs shfs smbfs sockfs spufs sshfs subfs supermount sysfs tmpfs ubifs udf usbfs vboxsf vperfctrfs"
    
    # Paths which are pruned from updatedb database
    PRUNEPATHS="/tmp /var/tmp /var/cache /var/lock /var/run /var/spool"
    
    # Folder names that are pruned from updatedb database
    PRUNENAMES=".git .hg .svn CVS"
    
    # Skip bind mounts.
    PRUNE_BIND_MOUNTS="yes"
  3. Running $ sudo updatedb then reduced it and saves a huge amount of time indexing all of those files (just went from 800MB to 1.6MB in my case).

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.