Hate to bother again, but I've got a situation I've been dealing with for a while. Nobody has offered much by way of troubleshooting and I can't seem to get it handled myself.
The issue is that one of my machines is running out of space on the /var partition. My first inclination was to believe that a log file was soaking up all the space because something was wrong. I found /var/log/kernel/info.log filling up despite log rolling.
When I follow the kernel info.log I see acres and acres of this:
- Code: Select all
Jan 6 21:34:14 adamsmdk kernel: [453261.815234] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=53079 DF PROTO=TCP SPT=1015 DPT=2049 WINDOW=1444 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453261.815329] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=180 TOS=0x00 PREC=0x00 TTL=64 ID=53080 DF PROTO=TCP SPT=1015 DPT=2049 WINDOW=1444 RES=0x00 ACK PSH URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453261.816483] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=180 TOS=0x00 PREC=0x00 TTL=64 ID=53081 DF PROTO=TCP SPT=1015 DPT=2049 WINDOW=1444 RES=0x00 ACK PSH URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453261.817148] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=180 TOS=0x00 PREC=0x00 TTL=64 ID=53082 DF PROTO=TCP SPT=1015 DPT=2049 WINDOW=1444 RES=0x00 ACK PSH URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453261.848753] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x00 PREC=0x00 TTL=64 ID=53083 DF PROTO=TCP SPT=1015 DPT=2049 WINDOW=1444 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453261.938729] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC=0x00 TTL=64 ID=28077 DF PROTO=TCP SPT=52776 DPT=2222 WINDOW=1444 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453261.999649] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC=0x00 TTL=64 ID=28078 DF PROTO=TCP SPT=52776 DPT=2222 WINDOW=1444 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.094869] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC=0x00 TTL=64 ID=28079 DF PROTO=TCP SPT=52776 DPT=2222 WINDOW=1444 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.181492] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=88 TOS=0x10 PREC=0x00 TTL=64 ID=6949 DF PROTO=TCP SPT=42530 DPT=2222 WINDOW=17866 RES=0x00 ACK PSH URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.182593] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC=0x00 TTL=64 ID=6950 DF PROTO=TCP SPT=42530 DPT=2222 WINDOW=17866 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.190160] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC=0x00 TTL=64 ID=28080 DF PROTO=TCP SPT=52776 DPT=2222 WINDOW=1444 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.281262] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=255.255.255.255 LEN=140 TOS=0x00 PREC=0x00 TTL=64 ID=18656 DF PROTO=UDP SPT=21327 DPT=21327 LEN=120
Jan 6 21:34:14 adamsmdk kernel: [453262.281287] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=ff:ff:ff:ff:ff:ff:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=255.255.255.255 LEN=140 TOS=0x00 PREC=0x00 TTL=64 ID=18657 DF PROTO=UDP SPT=21327 DPT=21328 LEN=120
Jan 6 21:34:14 adamsmdk kernel: [453262.283487] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=88 TOS=0x10 PREC=0x00 TTL=64 ID=6951 DF PROTO=TCP SPT=42530 DPT=2222 WINDOW=17866 RES=0x00 ACK PSH URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.284543] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC=0x00 TTL=64 ID=6952 DF PROTO=TCP SPT=42530 DPT=2222 WINDOW=17866 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.318736] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC=0x00 TTL=64 ID=28081 DF PROTO=TCP SPT=52776 DPT=2222 WINDOW=1444 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.380712] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC=0x00 TTL=64 ID=28082 DF PROTO=TCP SPT=52776 DPT=2222 WINDOW=1444 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.571238] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC=0x00 TTL=64 ID=28083 DF PROTO=TCP SPT=52776 DPT=2222 WINDOW=1444 RES=0x00 ACK URGP=0
Jan 6 21:34:14 adamsmdk kernel: [453262.605496] BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=88 TOS=0x10 PREC=0x00 TTL=64 ID=6953 DF PROTO=TCP SPT=42530 DPT=2222 WINDOW=17866 RES=0x00 ACK PSH URGP=0
Journalctl shows this:
- Code: Select all
an 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=180 TOS=0x00 PRE
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=180 TOS=0x00 PRE
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=120 TOS=0x10 PRE
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x00 PREC
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:52 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:00:19:d1:71:dd:e7:08:00 SRC=192.168.1.105 DST=192.168.1.100 LEN=172 TOS=0x00 PRE
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:00:19:d1:71:dd:e7:08:00 SRC=192.168.1.105 DST=192.168.1.100 LEN=172 TOS=0x00 PRE
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:00:19:d1:71:dd:e7:08:00 SRC=192.168.1.105 DST=192.168.1.100 LEN=172 TOS=0x00 PRE
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:00:19:d1:71:dd:e7:08:00 SRC=192.168.1.105 DST=192.168.1.100 LEN=172 TOS=0x00 PRE
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=88 TOS=0x10 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=136 TOS=0x10 PRE
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:bc:5f:f4:b7:14:5a:08:00 SRC=192.168.1.103 DST=192.168.1.100 LEN=176 TOS=0x00 PRE
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:bc:5f:f4:b7:14:5a:08:00 SRC=192.168.1.103 DST=192.168.1.100 LEN=52 TOS=0x00 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:bc:5f:f4:b7:14:5a:08:00 SRC=192.168.1.103 DST=192.168.1.100 LEN=176 TOS=0x00 PRE
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:00:19:d1:71:dd:e7:08:00 SRC=192.168.1.105 DST=192.168.1.100 LEN=180 TOS=0x00 PRE
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:00:19:d1:71:dd:e7:08:00 SRC=192.168.1.105 DST=192.168.1.100 LEN=52 TOS=0x00 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:00:19:d1:71:dd:e7:08:00 SRC=192.168.1.105 DST=192.168.1.100 LEN=180 TOS=0x00 PRE
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:bc:5f:f4:b7:14:5a:08:00 SRC=192.168.1.103 DST=192.168.1.100 LEN=52 TOS=0x00 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:00:19:d1:71:dd:e7:08:00 SRC=192.168.1.105 DST=192.168.1.100 LEN=52 TOS=0x00 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=52 TOS=0x10 PREC
Jan 06 21:34:53 adamsmdk kernel: BANDWIDTH_IN:IN=wlp3s0 OUT= MAC=7c:c3:a1:b5:d4:41:40:16:7e:22:a1:ca:08:00 SRC=192.168.1.106 DST=192.168.1.100 LEN=88 TOS=0x10 PREC
Not identical, but close enough to see that something is pushing a bunch of miscellaneous information into my logs.
I've snooped around the ath9k driver that drives the wifi card, but I wasn't able to make anything of the information I found.
I suspected it was iptables at first, but I don't believe that it is. It persists whether iptables is running or not. Having said that, I can't figure out where it's coming from or how to stop it. Is it debug info from somewhere around the network stack? It seems benign, but my /var partition is filling up.
Any ideas?
Thanks.
Mark
