by jiml8 » Sep 25th, '20, 22:15
If I understand your question the way you intend it, you are confused.
Xorg is a graphical desktop server. It is the basic component upon which all of the display managers, and window managers and decorations and so forth depend in order to put up the display that you see.
The Nvidia driver is a driver for your particular video hardware. It exists at a lower level in the system than Xorg does, and Xorg uses the nvidia driver to access and control your video hardware.
So, you need both.
The nouveau driver is an open source driver that works with xorg and with Nvidia video cards if you don't want to use the Nvidia driver, which is closed source. If this is what you want to remove, you need to blacklist it and install the Nvidia driver.