Mapping all possible physical theories

Young people should get some clue about all possibilities, avoid dead ends

I am kind of – albeit not obsessively – observing young, emerging physicists who have a chance to bring a new significant conceptual development or "revolution" to physics. It's hard to see whether such a future "revolution" will be an accidental side effect of some "modest" technical work or whether it will arise from some people's attempts to think really deeply and conceptually.

It seems to me that on this planet, the number of people below 30 who have independently understood the structure of the "space of ideas" and possible theories and who "really know" why there isn't any alternative to string theory as a theory of quantum gravity, to pick a major example, is at most "a few dozens". There are a few hundred young people who have worked on string theory as well but they were made to do so and they're not really "leaders" of the collaborations. Also, they may easily "change their mind" with a new boss.




A young person's journey to the cutting edge of theoretical physics is neither easy nor straightforward. One may try to take shortcuts and avoid many streets that look like dead ends. On the other hand, it may always turn out to be useful in the future if you have spent some time (hopefully limited time) in such a dead end. Some of the objects found in such a dead end may reappear in places that are not dead ends and they may be useful.

Another reason why dead ends may be useful is that if you get familiar with the "basic dead ends" and perhaps a larger (but still limited) number of dead ends in the foundations of physics, you may get immune against a much larger number of dead ends that lure conceptual researchers when they get to a more advanced stage of research.

Possible classical theories

When I was 4 or so, I started to think about the world as a theoretical physicist. It means that I became certain that the world is isomorphic to a mathematical model – yes, unfortunately, that paradigm was only good for classical physics at that time. The task was to find out the exact mathematical model that describes the real world.

The first category of models involved equations governing "where the matter is and where it isn't". The basic observables in these models were functions\[

f: \RR^{3+1}\to\{0,1\}.

\] For each point of the space and at each moment of time, you either specify whether "matter is there" or "matter is not there". There are different materials – and different colors of matter etc. – so one possibility to guarantee that the theory describes all of them would be to enhance the set \(\{0,1\}\) to a larger set. For example, the set could have five elements, corresponding to classical elements. (Yup, it's the same word!)

Alternatively, different materials could arise from the binary data but the microscopic composition of a material could involve both \(0\) as well as \(1\) in different patterns. This idea was "modern" and indeed, this idea is recycled in Nature many times. Nature doesn't need too many elementary particle species or too many different indivisible "atoms": it may produce them by various combinations of a smaller number of elementary building blocks. String theory is the most principled theory of this sort because even elementary particles are made out of the same string, one kind of a fundamental object.

If you identify the world with the information about the regions where matter is present and where matter is absent, you should hope that these regions are separated by simple enough boundaries. These boundaries may be parameterized by \(2+1\) parameters – the coordinates labeling domain walls – and the functions that determine the shape of these domain walls may be subject to dynamical equations. In some sense, this kindergarten model of mine was a model of open D3-branes.

Needless to say, soon afterwords, I was exposed to classical mechanics with point masses. They approximately describe the motion of celestial bodies and other things. The objects' coordinates are functions of time, \(x_i(t)\), that may be required to obey some differential equations. One may spend years with classical mechanics but the previous sentence is enough for my current purposes.

The identity of the differential equations that would govern the coordinates is unknown. In fact, all possible forms of the potential energy etc. seem equally justified. There's no principle that would pick the right functions among the \(\infty^\infty\) candidates. From such a broader viewpoint, classical mechanics is utterly unpredictive. However, you may try to guess the right functions – which may be simple enough – and voilà, some of the simplest ones will agree with the experiments.

Classical mechanics may be applied to the case of very many mass points. It may also be generalized to classical field theory whose basic time-dependent variables are fields \(\phi_j(x,y,z,t)\) that depend not only on time but also on space. In some sense, the extra independent arguments \(x,y,z\) play an analogous role to the index \(i\) in \(x_i(t)\) except that they're continuous: they increase the number of dynamical variables. These functions \(\phi_j(x,y,z,t)\) must obey partial differential equations. While \((x,y,z)\) are analogous to \(i\) when it comes to imagining what depends on what, you may also see an analogy between \((x,y,z)\) and \(t\). In fact, special relativity postulates a symmetry, the Lorentz symmetry, between these four coordinates. All inertial coordinate systems (reference frames) are equally good.

Conceptually, these classical field theory models may also be extended to the framework of general relativity in which an arbitrary redefinition of coordinates \((x,y,z,t)\) is allowed and in each set of coordinates, the physical laws are required to have the same simple form (in principle). Such theories have some redundancies but the stuff isn't really conceptually new. The real world is still identified with a mathematical model involving functions of many coordinates that are required to obey partial differential equations.

In all these models, time \(t\) is a continuous variable. Because of the Lorentz symmetry that you ultimately get certain about and whose importance you should appreciate, the continuity of \(t\) implies the continuity of the spatial coordinates \((x,y,z)\), too. It's useful to spend some time with candidate classical descriptions of Nature in which the time \(t\) is modeled by a discrete, not continuous, quantity. But you should do so critically, not mindlessly. When you do so, you will realize that all these models are deeply contrived and almost certainly wrong. Continuous symmetries become impossible if the spacetime coordinates are discrete in any sense. The absence of continuous symmetries will mean that an important principle to constrain the laws will be missing. None of the "real physical problems" of renormalizability or short-distance divergences (in classical physics or, later, quantum physics) would be solved by the discreteness, either. The discreteness is just a regularized definition of a theory you started with and the real problem signaled by the divergences is the infinite number of undertermined parameters (coefficients in front of non-renormalizable couplings).

Discrete physics is a major dead end where tons of young and not-so-young people get stuck. Once you accumulate enough experience from other streets and avenues, it is totally obvious to you that they won't get anywhere if they keep on spending months or years on this direction. However, they haven't really been "elsewhere" which often prevents them from seeing the essential features of theories with a continuous time that their discrete theories simply can't reproduce. This ignorance, usually combined with excessive stubbornness, is a dangerous mix.

That was the last moment I spent with the idea that the time could refuse to be "fundamentally continuous". All discrete things in Nature are ultimately derived, emergent. The fundamental laws of physics must be continuous, and the more they are continuous, the more fundamental they seem to be.

When we switched from classical mechanics to classical field theory, we made the mathematical objects conjectured to match the world more complicated. Instead of functions of one variable, we had functions of several variables. We may also consider functions of infinitely many variables, e.g. functionals\[

Z[f(x,y,z)]\in\RR.

\] For each function of the coordinates \((x,y,z)\), you choose a real number. That's what a functional means. If such an object fundamentally defined a classical theory, this theory would contradict locality. It's just too complicated. Generic functionals of this sort describe voodoo-like objects that affect several places in space simultaneously. The object \(Z\) doesn't seem to describe isolated objects. Let me tell you that once you work with a quantum theory, (complex-valued) functionals may describe the state vector in quantum field theory. The voodoo character goes away in this quantum interpretation (and only in the quantum interpretation) because the functionals' being nonzero at several widely separated places means that one OR the other possibility is realized – but both of these possibilities may still correspond to properties of local objects (the functional is a sort of probability distribution).

You could also consider even more complicated objects, "functional-als", which assign a real or complex number to each functional.\[

\Upsilon(Z[f(x,y,z)]).

\] In some sense, these functional-als are one of the possible descriptions of state vectors in string field theory, a definition of string theory that tries to emulate a quantum field theory with many fields. And you could go on and on and on, functional-al-al-al-als, but Al has already gotten $100 million in donation from the Qatari oil magnates so I don't want to add too many Al's here.

So far I was considering classical theories of physics. There is an objective model that is equivalent to Nature. It's important for every modern physicist who doesn't want to be "completely stuck" about basic issues to understand that this whole classical framework, regardless of its huge number of possibilities and various diverse and "clever" modifications and generalizations you could propose, is fundamentally wrong. As argued in hundreds of articles on this blog, there isn't any objective mathematical function or functional-al-al-al that would explain all observations we make.

Instead, you have pick the observations you make – the input information – translate it to a state vector or a density matrix, apply the transformations dictated by quantum mechanics, and you obtain probabilistic predictions for any meaningful statement about an observable in the future. This quantum framework is entirely different from the classical one and it's the right one.

Even without experimental tests, a good theorist should be able to see that the quantum framework is more natural, more general, and leads to more constrained theories. It's simply consistent to identify observables with mutually non-commuting operators. For this reason, it is an "extremely special and therefore unlikely assumption" that all observables commute with each other. This situation "everything commutes with everything else" may appear in restricted situations and in the classical limit. And the classical limit is a very special, "infinitesimal" subset of quantum mechanics. The constant \(\hbar\) is simply not equal to zero. Its being exactly zero is infinitely unlikely, even a priori. The quantum theory is the primary one and the classical one is at most an approximation valid in a limit.

In classical field theories, the functions encoding the potential energy and other things could have been arbitrary. In quantum field theory, they're tightly constrained. Perturbatively speaking, the Lagrangians have to be at most polynomials of the fourth degree (let me not go into the details here) for the theory to be renormalizable (i.e. consistent and predictive up to arbitrarily high energies, if I use a physical language focusing on the implications of the adjective). The number of coefficients is finite. The number of parameters is finite.

It's very important for a modern physicist to understand that quantum mechanics is fundamentally different from classical physics and it has different answers to many questions. Even questions that look "obvious" in classical physics may become tough to prove in quantum physics and some of these claims become just approximations. Nevertheless, quantum mechanics works. And its postulates can't really be "deformed" in any way. For example, the linearity required from the operators is fundamentally equivalent to (or at least equivalent to something that is similar to) addition formulae for probabilities of "A or B" etc. There isn't any logic or probability calculus where the probabilities would add up non-linearly. Analogously, there isn't any non-linear deformation of quantum mechanics worth considering as a theory of physics, and so on.

Once you spend some time with the research – but not a mindless one – of possible frameworks that are neither classical nor quantum, you will realize that classical physics and quantum physics are the only two truly sensible options and the former is known to be wrong. If you search for a theory of Nature, you're searching for a quantum theory.

It means that you have to have a Hilbert space – well, all infinite-dimensional Hilbert spaces are really unitarily equivalent to each other so at this level, there's no freedom. The diversity only emerges once you equip the Hilbert space with natural or interesting observables – linear operators – and one/several of them, e.g. the Hamiltonian (or the unitary S-matrix/evolution operator(s)) – has/have to encode the dynamical evolution in time.

Most of the often studied quantum theories may be obtained by a "straightforward" procedure of quantizing a classical system with an action. In this subset, the search for a quantum theory is equivalent to the search for a classical theory combined with the procedure of quantization. However, you should never forget that the procedure of quantization changes the predictions and the character of the theory. It changes its "moduli spaces", in some cases. It introduces nonzero commutators, quantization conditions, and monodromies. UV divergences and even anomalies appear that make some classical theories inconsistent at the quantum level. Some classical theories refuse to be unique as "generators" of quantum theories – there may be several quantum theories with the same classical limit, and so on.

As I said, the convoluted mathematics needed to calculate things within the quantum framework constrains the possible theories. In some sense, the space of consistent quantum theories is much more special than the space of consistent classical theories – even though some people think that there is a one-to-one correspondence whose map is kind of "straightforward". It's not that straightforward. If you study the space of possible consistent quantum theories, it's a different problem than the problem to map out possible classical theories. One must be aware of these differences. One must think quantum.

The constrained character of the quantum theories becomes really intense once you demand that the theory also contains gravity, approximately in the sense of the general theory of relativity. In this case, you may classify all possible candidate classical theories that could describe such a thing and quantize them, to see whether gravity really emerges. Most of these theories will be crippled by short-distance divergences, anomalies, and other inconsistencies. All "constructible" candidates that you may find and that will work will turn out to be equivalent to string theory in one of its incarnations. This conclusion can't be obvious from the beginning. People couldn't have assumed it. People hadn't known it 45 years ago. But once you really master the "city" with all the possible streets, avenues, and dead ends, you will see that my claim is right. All roads lead to string theory.

However, there is still the possibility that the right ultimate theory is not "constructible" in the sense that its "action" would be some functional of some basic fields which are (quantized) functions or functionals or functional-als, and so on. It may be some theory that determines its own structure. In physics, this possibility – and the research programs attempting to change the mere possibility to an operational theory – is known as "bootstrap" because the theory is apparently capable of "pulling itself out by its bootstraps". Heisenberg was an early proponent of this idea and it became very popular in the late 1960s when the strongly interacting "hadrons" started to resemble a zoo that couldn't possibly be "constructed".

If Nature is fundamentally described by a bootstrap-like theory, you can't really map the possibilities in advance. Instead, you must be lucky, find the mysterious principle behind everything, and verify that it is indeed correct. The theory doesn't have to be based on any particular set of functions or functionals or anything else in our sequence. It may be based on mathematical objects of some brutally generalized, vague type that only acquire their identity with a clear name once you solve some mathematical conditions written in terms of maths we can't really comprehend today. These conditions don't have to be any particular, pre-determined differential equations or anything of the sort.

(Two-dimensional conformal field theories – ones relevant for perturbative string theory, among other things – are a "restricted" example of a success of the bootstrap paradigm but they're still "too constructive" relatively to the hypothetical ultimate "full-fledged bootstrap" theories that should really talk about some "consistency" only.)

However, even if string/M-theory is most universally described in this bootstrap way, it doesn't mean that the "constuctive" approaches to find the theory are useless. Quite on the contrary, even if one has a bootstrap theory whose fundamental rules can't be written down in terms of particular functions and/or evolution equations depending on particular potential-energy-like functions, it's always likely that particular environments/solutions (points in its moduli space or configuration space) allowed by this theory will exist in the "extreme limits" and these extreme limits typically do admit "constructible" definitions. And that's what we seem to see in the case of string/M-theory, too. Its extreme limits, such as the weakly coupled limits, admit various types of perturbative and low-energy and other expansions while the "complete bulk of the configuration space" dealing with "generical compactifications and/or cosmology" remains a bit ill-defined and mysterious.

If a bootstrap-like definition of string/M-theory exists, it would be great to find it but you shouldn't forget that we don't really know whether it exists. No one is going to guarantee for you that if you spend years or decades by this pursuit, the investment has to be repaid.

Quite generally, I want to say that as young physicists are getting mature, they must increasingly rely on assumptions they have verified or even "rediscovered" by theselves. Too many young people – well, a big majority – still works on research programs whose rules and limits were defined by someone else. For example, someone decides that quantum gravity is just the addition of some "hats with a nice shape" to the classical observables and we should find it. Dozens if not hundreds of people then spend time by the futile attempts to describe the real world by something like loop quantum gravity, causal dynamical triangulations, spin foams, and several other hopeless proposals like that. The young people going into these programs forget – or don't want to know – that their leaders and advisers aren't infallible Gods. They rely on lots of assumptions about the hypothetical theory that are unverified or, if you verify them, that are demonstrably wrong.

Of course, other (and sometimes the same) young people are doing the same thing in the case of string theory research, too. A difference is that string theory works so they were more lucky about their leaders or advisers. But at the end, a theoretical physicist who wants to get to the conceptual cutting edge must get rid of the dependence on the unreliable shoulders of others. He or she must map out the space of possible ideas in physics, see how large or small various neighborhoods of this space are, which of them are hopeless ghettos or bomb sites – and such bomb sites may be small as well as vast (a suburb's being "large" doesn't mean that it is not a bomb site) – and which of them deserve to be studied in greater detail or generalized.

It's important for a young scientist not to scream resolute conclusions about things that he or she hasn't independently verified. Claiming "I know something" even though in reality it's just "I want to be a partisan and to defend whatever some other people say" is just a deeply unscientific attitude to questions that one should avoid from the beginning, from the low age. For example, it's really bad if someone gets stuck in the trap of the – using a highly diplomatic language – shitty demagogues, crackpots, and assholes such as Lee Smolin and Peter Woit and the mindless movements that these scumbags created around them just in order to harm the proper research in physics. It's likely that most of the people caught in these cesspools will never make it to the fresh air again.

But there are many more dead ends in the world. And many beautiful and important oases that may remain unexploited by tourists if most tourists spend most of their time in deserts and cesspools.

And that's the memo.