Intel bug leaks memory

Here wizards, magicians, sorcerers and everybody can rest a bit and talk about anything they like.

Just remember to respect the rules.

Intel bug leaks memory

Postby jiml8 » Jan 3rd, '18, 20:17

This is about as ugly as it gets:
https://www.theregister.co.uk/2018/01/0 ... sign_flaw/

The article refers to a "redesign" of Linux (and other OS's) to "fix" the problem - which evidently can't be handled by a microcode update. It also says the fix will rather substantially slow the system.

What I want to know is whether AMD based systems will also be slowed down. This bug does not affect AMD processors, but given how intel-centric so much of computing is, will the patched kernel distinguish between processors and not cripple AMD?

...now we know how Intel got at least part of its speed advantage over AMD...
jiml8
 
Posts: 936
Joined: Jul 7th, '13, 18:09

Re: Intel bug leaks memory

Postby Germ » Jan 3rd, '18, 22:02

yea, definitely ugly. My guess is the fix would slow your processor down whether it's intel or AMD
User avatar
Germ
 
Posts: 301
Joined: Mar 30th, '11, 13:16
Location: Chelsea, Oklahoma USA

intel kernel page table isolation bug

Postby wintpe » Jan 4th, '18, 15:46

Im working on this, at my day job, im guessing some of you who are sysadmins are too

Doktor?

Anyways for those who are not, the register has a good artical

https://www.theregister.co.uk/2018/01/0 ... sign_flaw/

it will mean a new kernel at some point.

there are three bugs, the 1st is the biggest one, that affects all intel CPU's

the other two lesser so, that affect other CPU's

regards peter
Redhat 6 Certified Engineer (RHCE)
Sometimes my posts will sound short, or snappy, however its realy not my intention to offend, so accept my apologies in advance.
wintpe
 
Posts: 1073
Joined: May 22nd, '11, 17:08
Location: Rayleigh,, Essex , UK

Re: intel kernel page table isolation bug

Postby doktor5000 » Jan 4th, '18, 16:08

Well jiml8 already has a thread for that: viewtopic.php?f=5&t=12204 If you don't object I'll simply merge it in there.

So far I've seen a good summary and overview of the actual CVE numbers:
https://www.theregister.co.uk/2018/01/0 ... erability/
https://meltdownattack.com/

It's probably worth pointing out that if you're using some cloud service providers like Azure, Amazon AWS or Google then you should check with them if your servers are already patched.
Some have already been, and some are being patched over the upcoming weekend. It's important as the vulnerability also potentially allows attacks from virtualised guests on the hosts.
Cauldron is not for the faint of heart!
Caution: Hot, bubbling magic inside. May explode or cook your kittens!
----
Disclaimer: Beware of allergic reactions in answer to unconstructive complaint-type posts
User avatar
doktor5000
 
Posts: 14198
Joined: Jun 4th, '11, 10:10
Location: Leipzig, Germany

Re: Intel bug leaks memory

Postby wintpe » Jan 4th, '18, 17:03

nope dont object at all, simply did not notice jims post.

Redhat have issued an update today for it, in the guise of kernel-3.10.0-693.11.6.el7.x86_64.rpm and kernel-2.6.32-696.18.7.el6.x86_64.rpm plus all the associated kernel support stuff.

someone on the centos mailing list suggests theres a conflict between the nvidia driver and the KpTI bug fix in the kernel, incompatible gpl-only symbol cpu_tlbstate, so worth watching out for that as well when we come to test.

im also hoping this change is not necessary for amd, but if they simply change the way the kernel swaps modes to ensure isolation then it might.

also what happens in virtualisation space, does that mean double wammy.

regards peter
Redhat 6 Certified Engineer (RHCE)
Sometimes my posts will sound short, or snappy, however its realy not my intention to offend, so accept my apologies in advance.
wintpe
 
Posts: 1073
Joined: May 22nd, '11, 17:08
Location: Rayleigh,, Essex , UK

Re: Intel bug leaks memory

Postby doktor5000 » Jan 4th, '18, 17:23

wintpe wrote:also what happens in virtualisation space, does that mean double wammy.

Yes, as long as the host and the guest share kernel space it might potentially allow the guest access to the host.
Cauldron is not for the faint of heart!
Caution: Hot, bubbling magic inside. May explode or cook your kittens!
----
Disclaimer: Beware of allergic reactions in answer to unconstructive complaint-type posts
User avatar
doktor5000
 
Posts: 14198
Joined: Jun 4th, '11, 10:10
Location: Leipzig, Germany

Re: Intel bug leaks memory

Postby wintpe » Jan 5th, '18, 00:50

hi Jim
this may quash your concerns, and mine

https://www.phoronix.com/scan.php?page= ... le-x86-PTI


regards peter
Redhat 6 Certified Engineer (RHCE)
Sometimes my posts will sound short, or snappy, however its realy not my intention to offend, so accept my apologies in advance.
wintpe
 
Posts: 1073
Joined: May 22nd, '11, 17:08
Location: Rayleigh,, Essex , UK

Re: Intel bug leaks memory

Postby xboxboy » Jan 5th, '18, 10:44

It's a nasty one alright, especially the Intel meltdown issue.

My main desktop at home is an aging, intel, nvidia rig, that is becoming horribly slow with the modern internet. If it takes a 30% slowdown as some are suggesting, it will be practically unusable for a desktop.

Lot's of issues for cloud providers, I haven't seen anyone mention the power usage increase that will occur from needing faster hardware to provide the same level of service.

The main take away for most end users, make sure you have backups, secure passwords and update your software as soon and as often as possible..... We were all doing that weren't we? /s

PS: Big shout out to TMB, all other devs and testers that keep on top of this stuff.
xboxboy
 
Posts: 273
Joined: Jun 2nd, '13, 06:41

Re: Intel bug leaks memory

Postby wintpe » Jan 5th, '18, 12:17

Redhat has provided the following info today about the switches

Grub/kernel

noibrs noibpb nopti

on the fly
# mount -t debugfs none /sys/kernel/debug
# echo 0 > /sys/kernel/debug/x86/pti_enabled
# echo 0 > /sys/kernel/debug/x86/ibpb_enabled
# echo 0 > /sys/kernel/debug/x86/ibrs_enabled

should be the same in all kernels.

Xboxboy the performance degradation is very workload dependent, and i suspect most desktop users will not notice any change.

if you do compute or heavy database workload you may notice an impact.

CVE-2017-5753 (variant #1/Spectre) is a Bounds-checking exploit during branching. This issue is fixed with a kernel patch. Variant #1 protection is always enabled; it is not possible to disable the patches. Red Hat’s performance testing for variant #1 did not show any measurable impact.

CVE-2017-5715 (variant #2/Spectre) is an indirect branching poisoning attack that can lead to data leakage. This attack allows for a virtualized guest to read memory from the host system. This issue is corrected with microcode, along with kernel and virtualization updates to both guest and host virtualization software. This vulnerability requires both updated microcode and kernel patches. Variant #2 behavior is controlled by the ibrs and ibpb tunables (noibrs/ibrs_enabled and noibpb/ibpb_enabled), which work in conjunction with the microcode.

CVE-2017-5754 (variant #3/Meltdown) is an exploit that uses speculative cache loading to allow a local attacker to be able to read the contents of memory. This issue is corrected with kernel patches. Variant #3 behavior is controlled by the pti tunable (nopti/pti_enabled).

update, ive also checked re hypervisor doing a cache miss and vm, in the case of KTPI, our vmware guy here believes thats not the case for ESXI, but maybe for KVM, and is for fusion.

regards peter
Redhat 6 Certified Engineer (RHCE)
Sometimes my posts will sound short, or snappy, however its realy not my intention to offend, so accept my apologies in advance.
wintpe
 
Posts: 1073
Joined: May 22nd, '11, 17:08
Location: Rayleigh,, Essex , UK

Re: Intel bug leaks memory

Postby doktor5000 » Jan 6th, '18, 09:57

wintpe wrote:Redhat has provided the following info today about the switches

Grub/kernel

noibrs noibpb nopti

on the fly
# mount -t debugfs none /sys/kernel/debug
# echo 0 > /sys/kernel/debug/x86/pti_enabled
# echo 0 > /sys/kernel/debug/x86/ibpb_enabled
# echo 0 > /sys/kernel/debug/x86/ibrs_enabled

should be the same in all kernels.


You may want to add some context information what those are meant for, they are meant to disable the effects of the patches to alleviate possible performance degradation.
Some more information on the context: https://access.redhat.com/articles/3311301#overview-1

To put this into perspective, a normal user or will probably not notice any performance degradations at all, the 30% slowdown was only measured in some scenarios with specific middleware.
See e.g. https://www.phoronix.com/scan.php?page= ... 6pti&num=2 and look at the "real world" examples, like kernel compilation or x264/ffmpeg encode.
Cauldron is not for the faint of heart!
Caution: Hot, bubbling magic inside. May explode or cook your kittens!
----
Disclaimer: Beware of allergic reactions in answer to unconstructive complaint-type posts
User avatar
doktor5000
 
Posts: 14198
Joined: Jun 4th, '11, 10:10
Location: Leipzig, Germany

Re: Intel bug leaks memory

Postby wintpe » Jan 6th, '18, 11:37

if you read the CVE's above they detail that

we tested mysql at work, using sqlbench

we saw about 2% difference

regards peter
Redhat 6 Certified Engineer (RHCE)
Sometimes my posts will sound short, or snappy, however its realy not my intention to offend, so accept my apologies in advance.
wintpe
 
Posts: 1073
Joined: May 22nd, '11, 17:08
Location: Rayleigh,, Essex , UK

Re: Intel bug leaks memory

Postby wintpe » Jan 16th, '18, 12:29

for anyone who hasnt found this already

Code: Select all
git clone https://github.com/speed47/spectre-meltdown-checker.git
Initialized empty Git repository in /root/spectre-meltdown-checker/.git/
remote: Counting objects: 393, done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 393 (delta 16), reused 18 (delta 8), pack-reused 365
Receiving objects: 100% (393/393), 137.35 KiB, done.
Resolving deltas: 100% (231/231), done.
# cd spectre-meltdown-checker
[spectre-meltdown-checker]# sh ./spectre-meltdown-checker.sh
Spectre and Meltdown mitigation detection tool v0.31
 
Checking for vulnerabilities against running kernel ...............
CPU is ......
 
CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Checking count of LFENCE opcodes in kernel:  YES
> STATUS:  NOT VULNERABLE  (84 opcodes found, which is >= 70, heuristic to be improved when official patches become available)
 
CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigation 1
*   Hardware (CPU microcode) support for mitigation
*     The SPEC_CTRL MSR is available:  NO
*     The SPEC_CTRL CPUID feature bit is set:  NO
*   Kernel support for IBRS:  YES
*   IBRS enabled for Kernel space:  NO
*   IBRS enabled for User space:  NO
* Mitigation 2
*   Kernel compiled with retpoline option:  NO
*   Kernel compiled with a retpoline-aware compiler:  NO
> STATUS:  VULNERABLE  (IBRS hardware + kernel support OR kernel with retpoline are needed to mitigate the vulnerability)
 
CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Kernel supports Page Table Isolation (PTI):  YES
* PTI enabled and active:  YES
* Checking if we're running under Xen PV (64 bits):  NO
> STATUS:  NOT VULNERABLE  (PTI mitigates the vulnerability)
 
A false sense of security is worse than no security at all, see --disclaimer



that run was on a redhat 6 system, and ive anonymized some of the info.

however its a nice and easy test for people to check what status they are.

for example, my laptop is still not fixed, because intel has not released all the firmware, which means mageia cannot provide a fix for variant 2 yet on all systems.

while amd has released firmware updates, the fix is not yet in them, again mageia cant do anything about this, as the microcode is proprietary, ie not open source.

your mileage will therefore vary.

regards peter
Last edited by doktor5000 on Jan 17th, '18, 19:19, edited 1 time in total.
Reason: added code tags
Redhat 6 Certified Engineer (RHCE)
Sometimes my posts will sound short, or snappy, however its realy not my intention to offend, so accept my apologies in advance.
wintpe
 
Posts: 1073
Joined: May 22nd, '11, 17:08
Location: Rayleigh,, Essex , UK

Re: Intel bug leaks memory

Postby wintpe » Jan 17th, '18, 19:07

Redhat and Vmware announce withdrawal of CPU microcode update from their patches, due to customer feedback of instability/system crashes on some recent (last 5 years) CPU's, and strongly recommends that customers contact their hardware provider for the latest microprocessor firmware updates.

Also based on Vmware and Redhats withdrawal any HW vendors bios updates should also be suspect.

anyone experiencing instability since the microcode_ctl update or bios update may want to backout a bios update or microcode_ctl update.

regards peter
Redhat 6 Certified Engineer (RHCE)
Sometimes my posts will sound short, or snappy, however its realy not my intention to offend, so accept my apologies in advance.
wintpe
 
Posts: 1073
Joined: May 22nd, '11, 17:08
Location: Rayleigh,, Essex , UK


Return to The Wizards Lair

Who is online

Users browsing this forum: No registered users and 1 guest