What is an intuitive explanation of a gauge transformation/invariance?
Gauge invariance, or gauge symmetry, is certain kind of symmetry of physical systems. Skip the next three paragraphs if you are generally comfortable with the concept of symmetries in physics.
In general, symmetries guarantee that observable quantities will be identical when certain symmetry transformations are performed. For translational symmetry, the symmetry transformation is translation. For rotational symmetry, the symmetry transformation is rotation.
In Newtonian physics, we know that the gravitational potential energy between two bodies doesn't change if both bodies are translated by the same amount, or if the entire system is rotated, keeping the distance between the bodies constant but changing their relative orientation. What we have, then, is a system that exhibits both translational and rotational symmetry. These symmetries guarantee that the gravitational potential energy between two bodies at rest depends only on the distance between them, and no other information about their coordinates. This in turn guarantees that the force between the bodies is directed radially.
In general, symmetries constrain the nature of physical law. For example, translational invariance guarantees that linear momentum is conserved, and rotational invariance guarantees that angular momentum is conserved. The previous conclusions concerning gravitational forces and potential energies can also be reached by considering these conservation laws. But it all goes back to the symmetries.
So the symmetries of a physical theory give us a lot of useful information about the system's behaviour. Gauge symmetries are no exception.
A gauge symmetry is a symmetry under a gauge transformation. Gauge transformations are unique because they are in some sense local. They contrast with global symmetry transformations, like translation and rotation.
For when you perform a translational symmetry operation, you have to translate every particle and every field in the universe by the same amount. There are obvious observable consequences if the entire universe except for my body is suddenly translated 100 m downward... but if the entire universe including my body is suddenly translated 100 m downward, nobody will notice that anything happened. When you perform a rotational symmetry operation, you get to pick your axis of rotation, but every particle and field has to be rotated by the same angle.
But a gauge transformation is allowed to affect different points of space-time differently. It would be as if you were allowed to pick a different distance to translate each particle in the universe by, and somehow, everything kept going exactly the way it was before.
So obviously translational and rotational symmetries are not gauge symmetries. What then is an example of a real gauge symmetry?
Now I could give you an example directly, but it wouldn't really help you get an intuitive understanding if you don't already have one. I think the right way to get an intuitive understanding is to take a step back and think about symmetries passively rather than actively.
As we saw, translational symmetry means you can shift the entire system—but it also has a different meaning, which is that once you've written down the equations of motion, if you decide to change the origin of your coordinate system, and plug in the new coordinates, they still satisfy the equations of motion. Rotational symmetry can also be re-cast in this "passive" form: if you rotate your axes, and plug in the new coordinates, they still satisfy the equations of motion. Or, less verbosely: there is no privileged location in the universe, nor is there a privileged direction. You can choose any origin and any orientation for your axes, and you still get the same laws of physics. It's easy to see that passive and active symmetries are related. For if shifting a system doesn't change the system's behaviour, then no experiment performed within the system can determine the system's "absolute" coordinates. The "absolute" coordinates contain an unobservable degree of freedom—the freedom to choose the origin wherever we like. Same goes for the angular positions of all the particles in the system.
Now the simplest example of a gauge symmetry is actually the gauge symmetry of electrodynamics, but I'm going to pick a more complicated example because I think it's more intuitive in a sense. We know that in nature, quarks are "coloured". They can be red, green, or blue, or somewhere in between. We also know that the colours of quarks are related to some kind of "strong force" between them that binds them into protons, neutrons, and other kinds of particles. The colour of a quark is a bit like a point on a sphere in a three-dimensional complex "colour space" (which is unrelated to physical space). If it's on the x-axis it's red, if it's on the y-axis it's green, and if it's on the z-axis it's blue, but it can also be at some other point on the sphere. In general to specify the colour you give its three coordinates in this "colour space".
Here's the thing, though: red, blue, and green quarks of a given flavour, as well as all the mixed-up colour combinations, are in every respect identical to each other. We know quarks must have colour, and that quark colours are superpositions of three basis colours, but if we could just isolate a single quark [1] it would be arbitrary whether we called it red, green, blue, or a superposition. So before we can decide what a quark's colour is—its coordinates relative to colour space, we need to have three "reference colours" to form a basis, like axes in the colour space—much like how we need to pick a set of unit basis vectors to form a set of coordinate axes before we can measure a particle's position coordinates. We need to arbitrarily choose what colours we want to call red, green, and blue. But what's new here is that we actually have the freedom to choose a different set of "axes in colour space" at every point in space-time. When we choose our axes at every point in space-time, we are choosing a gauge.
That is, if I have a solution to the equations of motion for quarks, but then I decide that at every point in space-time I'm going to rotate my "colour axes", possibly by a different amount at every point, and recalculate each quark's colour coordinates accordingly, the new set of numbers is still going to satisfy the equations of motion. Just like how when I rotate my axes in real space, and recalculate all particles' coordinates, I still get a valid solution to the equations of motion. That is a gauge symmetry.
So gauge symmetry is symmetry under arbitrary choice of gauge, subject to certain conditions. For the colour symmetry, the condition is that only certain rotations of the colour axes are allowed, those belonging to a certain group called SU(3). We can summarize this as "SU(3) gauge symmetry".
A gauge symmetry may affect some particles but not others. This particular gauge symmetry affects quarks, but not electrons. We say quarks are charged under the gauge symmetry, but electrons are not. This particular charge is called colour charge: quarks are colour-charged. The electromagnetic interaction also has a gauge symmetry, which I chose not to talk about. When that particular gauge symmetry, electrons are transformed, quarks are transformed by a smaller magnitude, and neutrinos are unaffected. That's what it really means when we say the quark's electric charge is fractional, and the neutrino is electrically neutral.
Back to quarks and colours. At first this might sound patently false. Because, you might say, let's imagine two quarks, one at the origin, one located a metre to the right... say both of them are red. Now if we do a "global" colour transformation, maybe now what we'll see is two green quarks. But if we do a gauge transformation that affects different points differently, then maybe the quark at the origin is still red but the other one appears green, since we rotated our axes differently at that point. How can this be equivalent to the original configuration? After all, we can certainly compare colours— a difference in colours is directly observable.
And the answer is twofold. First, differences in colour only have direct physical significance at the same point. So before we can see any observable consequence of a red quark and a green quark being different, we have to move them together to the same point. Second, to every gauge symmetry there corresponds a gauge field [3]. Whenever you perform a gauge transformation that varies between points, you also have to transform the fields in the region over which the transformation varies, by a corresponding amount, which might also differ from point to point. The gauge field for the colour symmetry of the quarks is the gluon field. So that means if we start out with two red quarks and a gluon field that is zero everywhere, and then decide to do a gauge transformation that will make the quark at the origin appear red but the other quark appear green, then in our new gauge, not only do we see two quarks of different colours, but we also see a nonzero gluon field, with a component along the x-axis. Furthermore, we postulate that the laws of motion stipulate that whenever a quark travels through a gluon field, the gluon field may change its colour by an amount proportional to the line integral of the gluon field over the quark's trajectory.
So now in the before scenario, we can move the two red quarks together and see that they are the same colour. But in the after scenario, after the gauge transformation, we have a red quark and a green quark and a gluon field; and if you move the red quark toward the green quark, because it passes through the gluon field, it will turn green, so once both quarks have arrived at the same point, they'll appear to be the same colour, just like in the before scenario! Likewise, if you were to move the green quark toward the red one, it would pass the gluon field in the opposite direction, and experience the opposite colour rotation—so it would appear red when it reached the other quark. So we really do have gauge invariance—we can't detect the change in the gauge.
But wait, didn't the gluon field change, too? Ha! It turns out that gluon field itself is unobservable. Now you might think this is cheating. Perhaps it sounds like I'm saying, "hey, guess what, there's a symmetry transformation that makes it possible for me to levitate, but at the same time it creates an unobservable field that bends the light in front of your eyes so it looks like I'm still on the ground." [cue laughter]
However, it turns out that there is an object that can be constructed out of the gauge field that is observable, called the field strength tensor [7]. The field strength tensor determines the forces on charged particles (where by "charged" I mean "charged under the gauge symmetry", not necessarily "electrically charged"). So, the gauge field is not just some entity we postulate so we can have gauge invariance. Were that the case, gauge invariance would be meaningless. Rather, we can regard a gauge theory as having three ingredients:
1) a set of charged particles,
2) a gauge field, whose field strength tensor exerts a force on charged particles,
3) a gauge transformation that affects both 1) and 2)
where the gauge transformation changes the gauge fields in such a way that the field itself changes, but its field strength tensor is unaffected. Just as the gauge symmetry on the quarks is based on unobservable degrees of freedom inherent in "absolute" colour coordinates, the field also has unobservable degrees of freedom, so that you can perform a gauge transformation on the field that changes the field while leaving the observable quantity, its field strength tensor, invariant. When you perform a gauge transformation, these unobservable degrees of freedom change for both 1) and 2) [4]. The final result is indistinguishable: the forces on all particles stay the same.
I mentioned several paragraphs back that the symmetries of nature give us insight into the behaviour of physical systems. What is the significance of gauge symmetry, then? It's twofold. First, gauge symmetries generate conservation laws just like translational and rotational symmetry. The conservation law corresponding to a gauge symmetry is the conservation of that gauge symmetry's charge. Therefore, the gauge symmetry of quarks and gluons implies that colour charge is conserved (which it is, in nature). Second, when you write down a Lagrangian for the system, in order for the Lagrangian to be gauge invariant, you are forced to write down a coupling term between each kind of charged particle or field, and the gauge field. When you do this, effecting the variation of the Lagrangian to produce the equations of motion naturally generates a force proportional to the charge and the gauge field's field strength tensor.
More concretely—as previously alluded to, the force that acts on quarks and gluons is the strong interaction of the Standard Model. It turns out that once you know that the quarks and gluons have an SU(3) gauge symmetry, this implies that there is a force between quarks, and allows you to work out its basic character [5]. Likewise, all I really have to tell you about electromagnetism is that it's a U(1) gauge theory—that alone is enough information to work out Maxwell's equations. [6]
Small wonder that gauge invariance is considered one of the basic ingredients of our most fundamental physical theories.
[1] We can't actually isolate a single quark, but this detail is not important for this discussion.
[2] Again, just pretend we're allowed to isolate quarks. (In reality, an isolated quark would have an infinite amount of energy.)
[3] In quantum field theory, gauge fields are massless vector fields, and their quanta are gauge bosons, which include the photon and gluon.
[4] A complication: the gluon field is itself charged, so when you do a gauge transformation, you have to "rotate" the gluon field as well as add to it the four-gradient of a scalar. This detail won't concern us here.
[5] Although the phenomenological details depend on certain other quantities as well: the total number of flavours of quarks, the masses of the quarks, the value of the coefficient in the Lagrangian (a.k.a. the "coupling constant"), and so on.
[6] The gauge field for electromagnetism is the photon field, which is analogous to the four-potential, and its field strength tensor is the electromagnetic field tensor, which describes the electric and magnetic fields. So a gauge transformation in electrodynamics changes the four-potential but leaves the E and B fields invariant. Also, the gauge symmetry implies conservation of electric charge. All this is consistent with everything I said before.
[7] The expression for the field strength tensor is similar to the exterior derivative of the field. For an abelian gauge theory such as electrodynamics, it is exactly the exterior derivative. For a non-abelian gauge theory such as quantum chromodynamics, an extra term proportional to \(A \wedge A\) has to be added. The actual definition of the field strength tensor is the tensor made up of the commutators of the gauge covariant derivatives, see, e.g., Gluon field strength tensor
In general, symmetries guarantee that observable quantities will be identical when certain symmetry transformations are performed. For translational symmetry, the symmetry transformation is translation. For rotational symmetry, the symmetry transformation is rotation.
In Newtonian physics, we know that the gravitational potential energy between two bodies doesn't change if both bodies are translated by the same amount, or if the entire system is rotated, keeping the distance between the bodies constant but changing their relative orientation. What we have, then, is a system that exhibits both translational and rotational symmetry. These symmetries guarantee that the gravitational potential energy between two bodies at rest depends only on the distance between them, and no other information about their coordinates. This in turn guarantees that the force between the bodies is directed radially.
In general, symmetries constrain the nature of physical law. For example, translational invariance guarantees that linear momentum is conserved, and rotational invariance guarantees that angular momentum is conserved. The previous conclusions concerning gravitational forces and potential energies can also be reached by considering these conservation laws. But it all goes back to the symmetries.
So the symmetries of a physical theory give us a lot of useful information about the system's behaviour. Gauge symmetries are no exception.
A gauge symmetry is a symmetry under a gauge transformation. Gauge transformations are unique because they are in some sense local. They contrast with global symmetry transformations, like translation and rotation.
For when you perform a translational symmetry operation, you have to translate every particle and every field in the universe by the same amount. There are obvious observable consequences if the entire universe except for my body is suddenly translated 100 m downward... but if the entire universe including my body is suddenly translated 100 m downward, nobody will notice that anything happened. When you perform a rotational symmetry operation, you get to pick your axis of rotation, but every particle and field has to be rotated by the same angle.
But a gauge transformation is allowed to affect different points of space-time differently. It would be as if you were allowed to pick a different distance to translate each particle in the universe by, and somehow, everything kept going exactly the way it was before.
So obviously translational and rotational symmetries are not gauge symmetries. What then is an example of a real gauge symmetry?
Now I could give you an example directly, but it wouldn't really help you get an intuitive understanding if you don't already have one. I think the right way to get an intuitive understanding is to take a step back and think about symmetries passively rather than actively.
As we saw, translational symmetry means you can shift the entire system—but it also has a different meaning, which is that once you've written down the equations of motion, if you decide to change the origin of your coordinate system, and plug in the new coordinates, they still satisfy the equations of motion. Rotational symmetry can also be re-cast in this "passive" form: if you rotate your axes, and plug in the new coordinates, they still satisfy the equations of motion. Or, less verbosely: there is no privileged location in the universe, nor is there a privileged direction. You can choose any origin and any orientation for your axes, and you still get the same laws of physics. It's easy to see that passive and active symmetries are related. For if shifting a system doesn't change the system's behaviour, then no experiment performed within the system can determine the system's "absolute" coordinates. The "absolute" coordinates contain an unobservable degree of freedom—the freedom to choose the origin wherever we like. Same goes for the angular positions of all the particles in the system.
Now the simplest example of a gauge symmetry is actually the gauge symmetry of electrodynamics, but I'm going to pick a more complicated example because I think it's more intuitive in a sense. We know that in nature, quarks are "coloured". They can be red, green, or blue, or somewhere in between. We also know that the colours of quarks are related to some kind of "strong force" between them that binds them into protons, neutrons, and other kinds of particles. The colour of a quark is a bit like a point on a sphere in a three-dimensional complex "colour space" (which is unrelated to physical space). If it's on the x-axis it's red, if it's on the y-axis it's green, and if it's on the z-axis it's blue, but it can also be at some other point on the sphere. In general to specify the colour you give its three coordinates in this "colour space".
Here's the thing, though: red, blue, and green quarks of a given flavour, as well as all the mixed-up colour combinations, are in every respect identical to each other. We know quarks must have colour, and that quark colours are superpositions of three basis colours, but if we could just isolate a single quark [1] it would be arbitrary whether we called it red, green, blue, or a superposition. So before we can decide what a quark's colour is—its coordinates relative to colour space, we need to have three "reference colours" to form a basis, like axes in the colour space—much like how we need to pick a set of unit basis vectors to form a set of coordinate axes before we can measure a particle's position coordinates. We need to arbitrarily choose what colours we want to call red, green, and blue. But what's new here is that we actually have the freedom to choose a different set of "axes in colour space" at every point in space-time. When we choose our axes at every point in space-time, we are choosing a gauge.
That is, if I have a solution to the equations of motion for quarks, but then I decide that at every point in space-time I'm going to rotate my "colour axes", possibly by a different amount at every point, and recalculate each quark's colour coordinates accordingly, the new set of numbers is still going to satisfy the equations of motion. Just like how when I rotate my axes in real space, and recalculate all particles' coordinates, I still get a valid solution to the equations of motion. That is a gauge symmetry.
So gauge symmetry is symmetry under arbitrary choice of gauge, subject to certain conditions. For the colour symmetry, the condition is that only certain rotations of the colour axes are allowed, those belonging to a certain group called SU(3). We can summarize this as "SU(3) gauge symmetry".
A gauge symmetry may affect some particles but not others. This particular gauge symmetry affects quarks, but not electrons. We say quarks are charged under the gauge symmetry, but electrons are not. This particular charge is called colour charge: quarks are colour-charged. The electromagnetic interaction also has a gauge symmetry, which I chose not to talk about. When that particular gauge symmetry, electrons are transformed, quarks are transformed by a smaller magnitude, and neutrinos are unaffected. That's what it really means when we say the quark's electric charge is fractional, and the neutrino is electrically neutral.
Back to quarks and colours. At first this might sound patently false. Because, you might say, let's imagine two quarks, one at the origin, one located a metre to the right... say both of them are red. Now if we do a "global" colour transformation, maybe now what we'll see is two green quarks. But if we do a gauge transformation that affects different points differently, then maybe the quark at the origin is still red but the other one appears green, since we rotated our axes differently at that point. How can this be equivalent to the original configuration? After all, we can certainly compare colours— a difference in colours is directly observable.
And the answer is twofold. First, differences in colour only have direct physical significance at the same point. So before we can see any observable consequence of a red quark and a green quark being different, we have to move them together to the same point. Second, to every gauge symmetry there corresponds a gauge field [3]. Whenever you perform a gauge transformation that varies between points, you also have to transform the fields in the region over which the transformation varies, by a corresponding amount, which might also differ from point to point. The gauge field for the colour symmetry of the quarks is the gluon field. So that means if we start out with two red quarks and a gluon field that is zero everywhere, and then decide to do a gauge transformation that will make the quark at the origin appear red but the other quark appear green, then in our new gauge, not only do we see two quarks of different colours, but we also see a nonzero gluon field, with a component along the x-axis. Furthermore, we postulate that the laws of motion stipulate that whenever a quark travels through a gluon field, the gluon field may change its colour by an amount proportional to the line integral of the gluon field over the quark's trajectory.
So now in the before scenario, we can move the two red quarks together and see that they are the same colour. But in the after scenario, after the gauge transformation, we have a red quark and a green quark and a gluon field; and if you move the red quark toward the green quark, because it passes through the gluon field, it will turn green, so once both quarks have arrived at the same point, they'll appear to be the same colour, just like in the before scenario! Likewise, if you were to move the green quark toward the red one, it would pass the gluon field in the opposite direction, and experience the opposite colour rotation—so it would appear red when it reached the other quark. So we really do have gauge invariance—we can't detect the change in the gauge.
But wait, didn't the gluon field change, too? Ha! It turns out that gluon field itself is unobservable. Now you might think this is cheating. Perhaps it sounds like I'm saying, "hey, guess what, there's a symmetry transformation that makes it possible for me to levitate, but at the same time it creates an unobservable field that bends the light in front of your eyes so it looks like I'm still on the ground." [cue laughter]
However, it turns out that there is an object that can be constructed out of the gauge field that is observable, called the field strength tensor [7]. The field strength tensor determines the forces on charged particles (where by "charged" I mean "charged under the gauge symmetry", not necessarily "electrically charged"). So, the gauge field is not just some entity we postulate so we can have gauge invariance. Were that the case, gauge invariance would be meaningless. Rather, we can regard a gauge theory as having three ingredients:
1) a set of charged particles,
2) a gauge field, whose field strength tensor exerts a force on charged particles,
3) a gauge transformation that affects both 1) and 2)
where the gauge transformation changes the gauge fields in such a way that the field itself changes, but its field strength tensor is unaffected. Just as the gauge symmetry on the quarks is based on unobservable degrees of freedom inherent in "absolute" colour coordinates, the field also has unobservable degrees of freedom, so that you can perform a gauge transformation on the field that changes the field while leaving the observable quantity, its field strength tensor, invariant. When you perform a gauge transformation, these unobservable degrees of freedom change for both 1) and 2) [4]. The final result is indistinguishable: the forces on all particles stay the same.
I mentioned several paragraphs back that the symmetries of nature give us insight into the behaviour of physical systems. What is the significance of gauge symmetry, then? It's twofold. First, gauge symmetries generate conservation laws just like translational and rotational symmetry. The conservation law corresponding to a gauge symmetry is the conservation of that gauge symmetry's charge. Therefore, the gauge symmetry of quarks and gluons implies that colour charge is conserved (which it is, in nature). Second, when you write down a Lagrangian for the system, in order for the Lagrangian to be gauge invariant, you are forced to write down a coupling term between each kind of charged particle or field, and the gauge field. When you do this, effecting the variation of the Lagrangian to produce the equations of motion naturally generates a force proportional to the charge and the gauge field's field strength tensor.
More concretely—as previously alluded to, the force that acts on quarks and gluons is the strong interaction of the Standard Model. It turns out that once you know that the quarks and gluons have an SU(3) gauge symmetry, this implies that there is a force between quarks, and allows you to work out its basic character [5]. Likewise, all I really have to tell you about electromagnetism is that it's a U(1) gauge theory—that alone is enough information to work out Maxwell's equations. [6]
Small wonder that gauge invariance is considered one of the basic ingredients of our most fundamental physical theories.
[1] We can't actually isolate a single quark, but this detail is not important for this discussion.
[2] Again, just pretend we're allowed to isolate quarks. (In reality, an isolated quark would have an infinite amount of energy.)
[3] In quantum field theory, gauge fields are massless vector fields, and their quanta are gauge bosons, which include the photon and gluon.
[4] A complication: the gluon field is itself charged, so when you do a gauge transformation, you have to "rotate" the gluon field as well as add to it the four-gradient of a scalar. This detail won't concern us here.
[5] Although the phenomenological details depend on certain other quantities as well: the total number of flavours of quarks, the masses of the quarks, the value of the coefficient in the Lagrangian (a.k.a. the "coupling constant"), and so on.
[6] The gauge field for electromagnetism is the photon field, which is analogous to the four-potential, and its field strength tensor is the electromagnetic field tensor, which describes the electric and magnetic fields. So a gauge transformation in electrodynamics changes the four-potential but leaves the E and B fields invariant. Also, the gauge symmetry implies conservation of electric charge. All this is consistent with everything I said before.
[7] The expression for the field strength tensor is similar to the exterior derivative of the field. For an abelian gauge theory such as electrodynamics, it is exactly the exterior derivative. For a non-abelian gauge theory such as quantum chromodynamics, an extra term proportional to \(A \wedge A\) has to be added. The actual definition of the field strength tensor is the tensor made up of the commutators of the gauge covariant derivatives, see, e.g., Gluon field strength tensor