[SOLVED] Startup issues with Clamd

[linuxdad]

I have used Clamd for about 7 years now. My new platform of Choice is Mageia, so I need to understand why this is an issue, and also how to resolve it.

The first thing I noticed is that the startup (init) script takes a good long time to complete, and leaves an error message:

Code: Select all

[root@ns init.d]# service clamd restart
Restarting clamd (via systemctl):  Job for clamd.service failed. See 'systemctl status clamd.service' and 'journalctl -n' for details.
                                                                [FAILED]

Reading through the clamd.log, we see the following info:

Code: Select all

Sat Dec  7 14:42:53 2013 -> Running as user defang (UID 503, GID 503)
Sat Dec  7 14:42:53 2013 -> Log file size limited to 20971520 bytes.
Sat Dec  7 14:42:53 2013 -> Reading databases from /var/share/clamav
Sat Dec  7 14:42:53 2013 -> Not loading PUA signatures.
Sat Dec  7 14:42:53 2013 -> Bytecode: Security mode set to "TrustSigned".
Sat Dec  7 14:43:05 2013 -> Loaded 3006535 signatures.
Sat Dec  7 14:43:09 2013 -> LOCAL: Unix socket file /tmpfs/clamd.socket
Sat Dec  7 14:43:09 2013 -> LOCAL: Setting connection queue length to 20
Sat Dec  7 14:43:09 2013 -> Limits: Global size limit set to 52428800 bytes.
Sat Dec  7 14:43:09 2013 -> Limits: File size limit set to 26214400 bytes.
Sat Dec  7 14:43:09 2013 -> Limits: Recursion level limit set to 10.
Sat Dec  7 14:43:09 2013 -> Limits: Files limit set to 500.
Sat Dec  7 14:43:09 2013 -> Limits: Core-dump limit is 0.
Sat Dec  7 14:43:09 2013 -> Archive support enabled.
Sat Dec  7 14:43:09 2013 -> Algorithmic detection enabled.
Sat Dec  7 14:43:09 2013 -> Portable Executable support enabled.
Sat Dec  7 14:43:09 2013 -> ELF support enabled.
Sat Dec  7 14:43:09 2013 -> Mail files support enabled.
Sat Dec  7 14:43:09 2013 -> OLE2 support enabled.
Sat Dec  7 14:43:09 2013 -> PDF support enabled.
Sat Dec  7 14:43:09 2013 -> HTML support enabled.
Sat Dec  7 14:43:09 2013 -> Self checking every 900 seconds.
Sat Dec  7 14:43:09 2013 -> ERROR: Can't save PID in file /var/run/clamd.pid
Sat Dec  7 14:43:09 2013 -> Listening daemon: PID: 5833
Sat Dec  7 14:43:09 2013 -> MaxQueue set to: 200
Sat Dec  7 14:43:09 2013 -> Set stacksize to 8454144
Sat Dec  7 14:58:09 2013 -> No stats for Database check - forcing reload
Sat Dec  7 14:58:10 2013 -> Reading databases from /var/share/clamav
Sat Dec  7 14:58:25 2013 -> Database correctly reloaded (3006535 signatures)
Sat Dec  7 15:13:25 2013 -> SelfCheck: Database status OK.
Sat Dec  7 15:28:25 2013 -> SelfCheck: Database status OK.
Sat Dec  7 15:43:25 2013 -> SelfCheck: Database status OK.


The interesting thing to notice is that the Clamd Daemon IS running, eventhough the startup indicates a failure.

Why?

Why is the Dameon having an issue writing the PID file?

Should I ignore this issue (I really don't want to).

Any suggestions?

[doktor5000]

Code: Select all

Sat Dec  7 14:42:53 2013 -> Running as user defang (UID 503, GID 503)
[...]
Sat Dec  7 14:43:09 2013 -> ERROR: Can't save PID in file /var/run/clamd.pid
Sat Dec  7 14:43:09 2013 -> Listening daemon: PID: 5833


The interesting thing to notice is that the Clamd Daemon IS running, eventhough the startup indicates a failure.

Why?

Why what? There can be simple errors which don't prevent the program from running but may put some constraints on normal functionality.

Why is the Dameon having an issue writing the PID file?

Should I ignore this issue (I really don't want to).

That's quite easy, just take a look:

Since some time /var/run is only a symlink to /run, but that's irrelevant here, only adding information for completeness' sake.

Code: Select all

[doktor5000@Mageia3 ~]$ ls -ald /var/run
lrwxrwxrwx 1 root root 6 Sep 28 20:18 /var/run -> ../run/

So what permission do we have on /run as normal user?

Code: Select all

[doktor5000@Mageia3 ~]$ ls -ald /run
drwxr-xr-x 32 root root 940 Dez  8 19:01 /run/
[doktor5000@Mageia3 ~]$ touch /run/clamd.pid
touch: cannot touch '/run/clamd.pid': Permission denied

As clamd runs under your UID/GID, you probably don't have the permissions to write there.

Related note, with the Mageia-packaged version of clamav there is no clamd service.

[isadora]

Hmm, let's follow this a bit further.

So the PID file is defined here:

Code: Select all

[root@ns ~]# grep pid /etc/clamd.conf
PidFile /var/run/clamav/clamd.pid

The Permissions are:

Code: Select all

[root@ns ~]# ls -la /var/run/clamav/clamd.pid
-rw-rw-r-- 1 defang defang 4 Dec  7 20:32 /var/run/clamav/clamd.pid

If clamd starts up as root, and then switches user to defang, what is preventing it from updating the PID file?

[doktor5000]

Where do you set those permissions? Because they will be gone after next reboot, as /run is tmpfs on a default installation.

Code: Select all

[doktor5000@Mageia3 SRPMS]$ ls -ald /run
drwxr-xr-x 34 root root 980 Dec  9 14:06 /run/
[doktor5000@Mageia3 SRPMS]$ df /var/run
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           2.0G  996K  2.0G   1% /run
[doktor5000@Mageia3 SRPMS]$ df /run
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           2.0G  996K  2.0G   1% /run

Anyways, cannot help you much further, as your clamav installation deviates too much from the Mageia packaged version of clamav, which doesn't even have clamd (what do you need/use that for in particular BTW?) so you need to figure that out on your own. Can only take a look at your logs and point out obvious issues that I can see, which is what I did.

[linuxdad]

Allow me to ask a simple question. If Clamd is started as root, by the new systemctl function, should it not be able to write to a file owned by either root, or by defang?

Do I need to SetUID on the clamd binary?

[doktor5000]

By common sense, it should be able to, yes.
And i'd definitely not hand over setuid/setgid permissions to any daemon just because something is not working, that's a really bad approach and should normally never be used.

EDIT: Just saw that there's a separate clamd package which I've totally missed, but it also works just fine here with the default config. OK, needed to stop/start clamd but now it looks just fine:

Code: Select all

[doktor5000@Mageia3 SRPMS]$ up clamd


    ftp://ftp-stud.hs-esslingen.de/pub/Mirrors/Mageia/distrib/3/x86_64/media/core/release/clamd-0.97.8-1.mga3.x86_64.rpm
clamd-0.97.8-1.mga3.x86_64.rpm von /var/cache/urpmi/rpms wird installiert                                                                                             
Vorbereiten …                    #####################################################################################################################################
      1/1: clamd                 #####################################################################################################################################
clamd.service is not a native service, redirecting to /sbin/chkconfig.
Executing /sbin/chkconfig clamd on
[doktor5000@Mageia3 SRPMS]$ sudo systemctl status clamd.service
clamd.service - LSB: An AntiVirus Server
          Loaded: loaded (/etc/rc.d/init.d/clamd)
          Active: inactive (dead)
          CGroup: name=systemd:/system/clamd.service

Code: Select all

[doktor5000@Mageia3 SRPMS]$ sudo systemctl stop clamd.service
[doktor5000@Mageia3 SRPMS]$ sudo systemctl status clamd.service
clamd.service - LSB: An AntiVirus Server
          Loaded: loaded (/etc/rc.d/init.d/clamd)
          Active: inactive (dead)
          CGroup: name=systemd:/system/clamd.service

Dec 09 15:23:54 Mageia3 systemd[1]: Stopped LSB: An AntiVirus Server.
[doktor5000@Mageia3 SRPMS]$ sudo systemctl start clamd.service
[doktor5000@Mageia3 SRPMS]$ sudo systemctl status clamd.service
clamd.service - LSB: An AntiVirus Server
          Loaded: loaded (/etc/rc.d/init.d/clamd)
          Active: active (running) since Mon, 2013-12-09 15:24:13 CET; 40s ago
         Process: 18226 ExecStart=/etc/rc.d/init.d/clamd start (code=exited, status=0/SUCCESS)
        Main PID: 18260 (clamd)
          CGroup: name=systemd:/system/clamd.service
                  └ 18260 clamd

Dec 09 15:24:13 Mageia3 clamd[18226]: Der Clam AntiVirus-Systemdienst wird gestartet: [  OK  ]
Dec 09 15:24:13 Mageia3 systemd[1]: Started LSB: An AntiVirus Server.

Testing scan:

Code: Select all

doktor5000@Mageia3 SRPMS]$ clamdscan /usr/share/doc/clamav/test/*
/usr/share/doc/clamav/test/clam.7z: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.arj: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-aspack.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.bin-be.cpio: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.bin-le.cpio: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.bz2.zip: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.cab: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam_cache_emax.tgz: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.chm: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.d64.zip: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.ea05.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.ea06.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.exe.binhex: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.exe.bz2: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.exe.html: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.exe.mbox.base64: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.exe.mbox.uu: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.exe.rtf: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.exe.szdd: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-fsg.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.impl.zip: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam_IScab_ext.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam_IScab_int.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam_ISmsi_ext.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam_ISmsi_int.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.mail: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-mew.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.newc.cpio: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-nsis.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.odc.cpio: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.ole.doc: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.pdf: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-pespin.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-petite.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.ppt: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.sis: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.tar.gz: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.tnef: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-upack.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-upx.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-v2.rar: OK
/usr/share/doc/clamav/test/clam-v3.rar: OK
/usr/share/doc/clamav/test/clam-wwpack.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam-yc.exe: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/clam.zip: ClamAV-Test-File FOUND
/usr/share/doc/clamav/test/Makefile: OK
/usr/share/doc/clamav/test/Makefile.am: OK
/usr/share/doc/clamav/test/Makefile.in: OK
/usr/share/doc/clamav/test/README: OK

----------- SCAN SUMMARY -----------
Infected files: 44
Time: 2.796 sec (0 m 2 s)

[linuxdad]

One by one, we are chipping away at the issues:

Code: Select all

[root@ns ~]# systemctl stop clamd.service
[root@ns ~]# systemctl start clamd.service
[root@ns ~]# systemctl status clamd.service
clamd.service - LSB: An AntiVirus Server
          Loaded: loaded (/etc/rc.d/init.d/clamd)
          Active: active (running) since Mon, 2013-12-09 09:51:18 EST; 5s ago
         Process: 7109 ExecStart=/etc/rc.d/init.d/clamd start (code=exited, status=0/SUCCESS)
        Main PID: 7135 (clamd)
          CGroup: name=systemd:/system/clamd.service
                  â 7135 clamd

Dec 09 09:51:02 ns.IT-Security-inc.com clamd[7125]: clamd daemon 0.97.8 (OS: linux-gnu, ARCH: i386, CPU: i586)
Dec 09 09:51:02 ns.IT-Security-inc.com clamd[7125]: Running as user defang (UID 503, GID 503)
Dec 09 09:51:02 ns.IT-Security-inc.com clamd[7125]: Log file size limited to 20971520 bytes.
Dec 09 09:51:02 ns.IT-Security-inc.com clamd[7125]: Reading databases from /var/share/clamav
Dec 09 09:51:02 ns.IT-Security-inc.com clamd[7125]: Not loading PUA signatures.
Dec 09 09:51:02 ns.IT-Security-inc.com clamd[7125]: Bytecode: Security mode set to "TrustSigned".
Dec 09 09:51:13 ns.IT-Security-inc.com clamd[7125]: Loaded 3009764 signatures.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7125]: LOCAL: Unix socket file /tmpfs/clamd.socket
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7125]: LOCAL: Setting connection queue length to 20
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Limits: Global size limit set to 52428800 bytes.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Limits: File size limit set to 26214400 bytes.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Limits: Recursion level limit set to 10.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Limits: Files limit set to 500.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Limits: Core-dump limit is 0.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Archive support enabled.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Algorithmic detection enabled.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Portable Executable support enabled.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: ELF support enabled.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Mail files support enabled.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: OLE2 support enabled.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: PDF support enabled.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: HTML support enabled.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Self checking every 900 seconds.
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Listening daemon: PID: 7135
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: MaxQueue set to: 200
Dec 09 09:51:17 ns.IT-Security-inc.com clamd[7135]: Set stacksize to 8454144
Dec 09 09:51:18 ns.IT-Security-inc.com clamd[7109]: Starting Clam AntiVirus Daemon: [  OK  ]
Dec 09 09:51:18 ns.IT-Security-inc.com systemd[1]: Started LSB: An AntiVirus Server.
[root@ns ~]#

[doktor5000]

Ok, but what did you do differently then before so that it works now?

[linuxdad]

I killed the running daemon (somehow it got stuck) with the kill command.

Then as displayed, I issued the systemctl stop clamd.service and then a start.

(I liked the service functionality so much more than the systemctl).

[doktor5000]

(I liked the service functionality so much more than the systemctl).

Yep, but the integrated journal output that comes with systemctl and also the child processes and stuff are also nice little things which SysVinit didn't provide out-of-the box.

[linuxdad]

Yep, but the integrated journal output that comes with systemctl and also the child processes and stuff are also nice little things which SysVinit didn't provide out-of-the box.

I understand, being a consultant there is rarely benefits without having a little pain. However, this updated was plagued with issues that made the update painful, at the least.